Set the file data to the bytes of a CSV and change the MIME type to CSV.
Set the file data to the bytes of JSON and change the MIME type to JSON.
Remove the file bytes and set a parsed data table.
Retrieve the file encoding.
Convert the bytes to a string using the detected encoding from GetEncoding.
A helper function to get a table from a byte array and MIME type, or if the data table has been assigned, it is returned instead.
Get a configuration value, even if it is encrypted. This method allows you to read configurations when writing plugins for the TransformClient Automation application.
key: Key of the value (e.g., "AppSettings:MyAPIKey").
data.SetDataAsCSV("id,name\n1,John Doe");data.SetDataAsJson("{\"id\":1,\"name\":\"John Doe\"}");data.SetAsDataTable(new DataTable());Encoding encoding = data.GetEncoding;string fileContent = data.BytesToString();DataTable dataTable = data.TableFromFile();string configValue = data.GetConfiguration("AppSettings:MyAPIKey");The DataQualityContext inherits from the Transform Context, to see the full list of methods, check out the page dedicated to those methods as well.
Process the data with a result, requirements function, and a callback for the data.
Parameters:
DataQualityValidationResult result: Result from the interface.
Func<bool> RequirementsFunction: Requirements function.
Action<DataQualityValidationResult, DataRow, long, ConcurrentBag<DataQualityValidationRow>> callback: Data callback.
int ChecksPerformedPerRow: How many checks were performed at each row?
Process the data with a result, requirements function, and a callback for the data.
Parameters:
DataQualityValidationResult result: Result from the interface.
Func<bool> RequirementsFunction: Requirements function.
Action<ConcurrentBag<DataQualityValidationRow>> callback: Data callback.
Calculate how many checks were performed on a list of associated sets.
Parameters:
string name: Name of the set.
int checksPerRow: How many checks per row (default is 1).
bool TransformOnly: If true, only transformed tables are returned (default is true).
Returns:
int: Number of checks performed.
int ChecksPerformedPerRow: How many checks were performed at each row?data.Process(result, RequirementsFunction, callback, ChecksPerformedPerRow);data.ProcessSet(result, RequirementsFunction, callback, ChecksPerformedPerRow);int checkCount = data.CheckCount_AssociatedSets("setName", checksPerRow, TransformOnly);This class represents the result of a data transformation process, encapsulating various properties and methods needed to handle transformed data, quality validation results, transformation report, and other related information.
Returns the result table as JSON.
Example:
Returns the result table as a compressed byte object.
Example:
Gets the filename from the map, calls MappingSpecification.GetCSVName.
Example:
Gets the filename from the map, calls MappingSpecification.GetXLSXName.
Example:
Gets the filename from the map, calls MappingSpecification.GetJSONName.
Example:
Converts the lookup maps back into a DataTable.
Example:
Generates a table for the missing maps information.
Example:
Runs segment validation, adding the results to DataQualityResults.
Parameters:
path (string): Path to import.
Example:
Runs segment validation, adding the results to DataQualityResults.
Parameters:
Segments (List<SegmentValidation.SegmentRaw>): List of raw segments.
Example:
Runs segment validation, adding the results to DataQualityResults.
Parameters:
Segments (byte[]): Byte array of compressed segments.
Example:
Writes the CSV file for the transformed file and appends the results to the transformation report.
Example:
Writes the XLSX file for the transformed file.
Parameters:
topColor (XLColor, optional): XL top band color if any header rows are added.
headerRows (int, optional): Number of header rows.
headerImageFile (byte[], optional): If provided, an image will be added at the top within the header rows.
Example:
Writes the JSON file for the transformed file.
Parameters:
formatting (Newtonsoft.Json.Formatting, optional): JSON formatting.
Example:
Generates the tables needed for a transformation report.
Example:
Generates the tables needed for a data quality report.
Example:
Produces a zip of the results of the transformation.
Parameters:
includeMap (bool, optional): Includes mapping file.
includeTransformed (bool, optional): Includes transformed file.
includeTransformReport (bool, optional): Includes transform report.
Example:
Produces a zip of the results of the transformation with branding information from a byte array.
Parameters:
includeMap (bool, optional): Includes mapping file.
includeTransformed (bool, optional): Includes transformed file.
includeTransformReport (bool, optional): Includes transform report.
Example:
The data table name (Transformation Group).
A data object ID assigned from the map used.
A list of data quality results.
The transformation report.
The resulting target table.
Lookup maps used in process.
The mapping specification used during the process.
The filename, if supplied, of the source file.
The time the transformation took.
Indicates if there are missing maps.
includeDQReport (bool, optional): Includes DataQualityReport.includeMissingMaps (bool, optional): Includes missing maps, if any.
IncludeLookupMaps (bool, optional): Includes used lookupValueMaps.
rptLogoPath (string, optional): If supplied, used as the branded logo header for reports.
rptHeaderRows (int, optional): Number of header rows to append.
rptTopColor (XLColor, optional): Header row solid color fill.
inputFileBytes (byte[], optional): Original file will also be included in the zip if supplied.
inputFileName (string, optional): Original file name if supplied.
format (CleanFileFormat, optional): Sets the file format of the transformed file.
includeDQReport (bool, optional): Includes DataQualityReport.includeMissingMaps (bool, optional): Includes missing maps, if any.
IncludeLookupMaps (bool, optional): Includes used lookupValueMaps.
rptLogo (byte[], optional): If supplied, used as the branded logo header for reports.
rptHeaderRows (int, optional): Number of header rows to append.
rptTopColor (XLColor, optional): Header row solid color fill.
inputFileBytes (byte[], optional): Original file will also be included in the zip if supplied.
inputFileName (string, optional): Original file name if supplied.
format (CleanFileFormat, optional): Sets the file format of the transformed file.
string jsonResult = transformationResult.AsJson();byte[] compressedResult = transformationResult.AsCompressedBytes();string csvFileName = transformationResult.GetCSVNameOfExportFile();string xlsxFileName = transformationResult.GetXLSXNameOfExportFile();string jsonFileName = transformationResult.GetJSONNameOfExportFile();DataTable lookupsTable = transformationResult.LookupsToTable();DataTable missingMapsTable = transformationResult.GetMissingMapsTable();transformationResult = transformationResult.Yardi_SegmentValidation("path/to/file");List<SegmentValidation.SegmentRaw> segments = new List<SegmentValidation.SegmentRaw>();
transformationResult = transformationResult.Yardi_SegmentValidation(segments);byte[] segments = new byte[0];
transformationResult = transformationResult.Yardi_SegmentValidation(segments);byte[] csv = transformationResult.WriteCSV();byte[] xlsx = transformationResult.WriteXLSX();byte[] json = transformationResult.WriteJSON();List<DataTable> transformationReportTables = transformationResult.GenerateTransformationReport();List<DataTable> dataQualityReportTables = transformationResult.GenerateDataQualityReport();byte[] zippedResults = transformationResult.ToZippedResults();byte[] zippedResultsWithBranding = transformationResult.ToZippedResults_Branding();Processes each row in the table using the provided filter predicate and data processor actions.
Parameters:
isValid: A function that returns a boolean indicating if the process is valid.
FilterPredicate: A function to filter input rows.
DataProcessor: An action to process each row.
Processes the entire table using the provided data processor action.
Parameters:
isValid: A function that returns a boolean indicating if the process is valid.
DataProcessor: An action to process the table.
Processes each row of the table by converting it to a target class. It sends non-null rows to the data processor.
Parameters:
isValid: A function that returns a boolean indicating if the process is valid.
DataProcessor: An action to process each row.
Filters rows in parallel based on the provided predicate.
Parameters:
predicate: A function to filter rows.
Checks if a column value in a row is null or empty.
Parameters:
row: The data row.
column: The column name.
Iterates over each column and uses the callback if the column type is valid.
Parameters:
row: The data row.
columns: A dictionary of column names and boolean values.
t: The type to validate.
Iterates over each column in the dictionary and casts the value to the specified type.
Parameters:
row: The data row.
columns: A dictionary of column names and boolean values.
FilterNulls: If true, no callback is given on null values.
Checks if the specified columns exist.
Parameters:
ColNames: The column names.
Checks if at least one of the specified column exists.
Parameters:
ColNames: The column names.
Checks if any columns exist of the specified types.
Parameters:
type: The types to check for.
Returns a list of columns that are of the specified types.
Parameters:
type: The types to check for.
Checks if a column exists.
Parameters:
name: The name of the column.
Gets the column name from the target column name.
Parameters:
name: The target field name.
Converts a data row to a specified class.
Parameters:
r: The data row.
UseTargetFieldName: If true, maps through the mapping specification.
Clean: Outputs if the row could be completely converted.
Converts the entire data table to a list of specified classes.
Parameters:
filterNull: If true, filters out null converted items.
Gets a value from a data row by the map target name and converts it to the specified type.
Parameters:
row: The data row.
name: The name of the target column.
val: Outputs the value.
Sets and validates a value of a row.
Parameters:
row: The data row.
column: The target column name.
val: The value to set.
Sets and validates a value of a row and generates a report if it fails.
Parameters:
row: The data row.
column: The target column name.
val: The value to set.
Validates that a new value is valid against the map restrictions for any given column.
Parameters:
columnName: The name of the column.
value: The value to validate.
Checks if all specified data sets are present.
Parameters:
TransformOnly (bool): If true, only transformed tables are returned. Default is true.
SourceOnly (bool): If true, only source (untransformed) tables are considered. Default is false.
Checks if a specified data set is present.
Parameters:
name (string): Name of the set.
TransformOnly (bool): If true, only transformed tables are returned. Default is true.
Retrieves all associated data sets by name.
Parameters:
name (string): Name of the set.
TransformOnly (bool): If true, only transformed tables are returned. Default is true.
Creates a data context from a set name.
Parameters:
name (string): Original table names from the set.
TransformOnly (bool): If true, only the transformed tables are considered.
Converts the first available set to an enumerable of a specified type.
Parameters:
T (type): Type to which the set will be converted.
originalName (string): Original name of the transform sets.
Iterates over each set looking them up by original name.
Parameters:
originalName (string): Original name.
Callback (Action): Callback per set.
FilterNulls: If true, no callback is given on null values.col: A callback function.
col: A callback function.uncleanColumnNames: Outputs the list of names of the columns that failed to convert.names (string[]): Names of the sets.Returns: A boolean indicating whether all specified sets are present.
SourceOnly (bool): If true, only source (untransformed) tables are considered. Default is false.Returns: A boolean indicating whether the specified set is present.
SourceOnly (bool): If true, only source (untransformed) tables are considered. Default is false.Returns: A list of DataTable objects for the specified sets.
SourceOnly (bool): If true, only source (untransformed) tables are considered. Default is false.Returns: A list of TransformDataContext objects for the specified sets.
TransformedOnly (bool): If true, only transformed tables are returned. Default is true.SourceOnly (bool): If true, only source (untransformed) tables are considered. Default is false.
filterNull (bool): If true, will filter out null rows. Default is false.
Returns: An enumerable of tuples containing the data of type T, a boolean indicating cleanliness, and a list of unclean column names.
TransformedOnly (bool): If true, only transformed tables are considered.SourceOnly (bool): If true, only source (untransformed) tables are considered. Default is false.
Func<bool> isValid = () => true;
Func<DataRow, bool> filterPredicate = row => row.Field<int>("ID") > 10;
Action<DataRow, long> dataProcessor = (row, index) => Console.WriteLine(row);
data.ProcessRows(isValid, filterPredicate, dataProcessor);Func<bool> isValid = () => true;
Action<DataTable> dataProcessor = table => Console.WriteLine(table.Rows.Count);
data.ProcessTable(isValid, dataProcessor);Func<bool> isValid = () => true;
Action<DataRow, bool, MyClass> dataProcessor = (row, clean, myClass) => Console.WriteLine(myClass);
data.ProcessRowsOfClass<MyClass>(isValid, dataProcessor);Func<DataRow, bool> predicate = row => row.Field<int>("ID") > 10;
var filteredRows = data.ParallelRowFilter(predicate);var isNullOrEmpty = data.ColumnNullOrEmptyFilter(row, "ColumnName");Dictionary<string, bool> columns = new Dictionary<string, bool> { { "Column1", true }, { "Column2", false } };
Type type = typeof(string);
Action<string, string, bool> callback = (colName, value, isNull) => Console.WriteLine($"{colName}: {value}");
data.EachColumnWithType(row, columns, type, callback);Dictionary<string, bool> columns = new Dictionary<string, bool> { { "Column1", true }, { "Column2", false } };
Action<string, string, bool> callback = (colName, value, isNull) => Console.WriteLine($"{colName}: {value}");
data.EachColumn<string>(row, columns, callback);bool allExist = data.RequiredColumns("Column1", "Column2");var columnsExist = data.RequiredColumnsOR("Column1", "Column2");bool columnsExist = data.AnyColumnsOfType(typeof(int), typeof(string));var columns = data.ColumnsOfType(typeof(int), typeof(string));bool exists = data.ColumnExists("ColumnName");string columnName = data.ColumnName("TargetFieldName");MyClass myClass = data.ToClass<MyClass>(row, out bool clean, out var uncleanColumnNames);var classList = data.ToClassList<MyClass>();bool success = data.GetValue<int>(row, "ColumnName", out int value);bool success = data.SetAndValidateValue(row, "ColumnName", value);bool success = data.SetAndValidateValueOrReport(row, "ColumnName", value);bool valid = data.ValidateValue("ColumnName", value);bool allSetsPresent = data.ContainsAssociatedSets(true, false, "set1", "set2");bool isSetPresent = data.ContainsAssociatedSet("set1", true);List<DataTable> sets = data.AssociatedSets("set1", true, false);List<TransformDataContext> contexts = data.DataContextFromSet("set1", true, false);var classData = data.SetToClass<MyClass>("set1", true, false, true);data.ForEachSet("set1", table => {
// Handle each table
}, true, false);