Dynamic Data Table
Dynamic?
Dyanmic Data Tables work similiarly to DataTables. The key difference being they are very loosely defined until the data is solved.
This allows for rows to be added anywhere without keeping up if that row is currently present.
Headers are automatically detected within the data set
Column types are solved, and support solving mesh types within a column.
Metrics are stored and calculated about each row and column.
They can easily be converted to fully structurally defined DataTable's at any point.
The example table
Let's take a look at a basic table with 3 columns, and 3 rows.
ID | Name | Age |
---|---|---|
1 | John | 25 |
2 | Smith | 30 |
3 | Jane | 29 |
To create this the standardized way using DataTable in the shortest possible way, here's how to recreate this table:
Here is how you would do the same thing with a dynamic data table:
Data Typing
In the above example, notice what's different between the two? The Data Types are not present! This is one of the key differences and the reason behind the Dyanmic in the class name.
We also do not have to worry about what rows we've added, or not added. They keep up with themselves.
DynamicDataTable
supports several additional "smart" features that allow it to process and solve the data while processing it. This is a huge advantage when loading data that is not a pristine export.
If you were to change row 3 in the first example you would generate an ArgumentException
:
Changing row 3 in a DynamicDataTable
will convert that column into a string:
Data types will be auto solved after N(default 1000) rows are added and will then be readjusted as rows are added.
This is the secret sauce behind the Excel Reader, it uses a DynamicDataTable
to solve and strip header content before load.
Header solve
What happens when your data is not on the first row? This is another big problem when dealing with a client file. There may be rows above the actual table and we don't need that information.
You can see it automatically solved the header row after calling .FinishDataLoad()
. The header, column data types, fill rates, data lengths and data type counts will all be solved and available.
Data Table methods
There are two primary methods for converting the dynamic table into a standard DataTable.
The .ToDataTable_ColumnsOnly()
version does not convert any data, only the schema of the table (the columns).
The .ToDataTable()
version has two optional parameters for splicing the table. If left at default values, the whole table is converted and returned.
CSV
You can easily create a CSV out of the data by calling .ToCSV()
. This is a handy method for quickly exporting data or debugging.
Static Values
You may add static values to the data table, they will be appended to the end of the columns load set.
"Special" Columns
The dynamic table comes with a few included "Special" columns that can be automatically added during the load.
Data Metrics
To retrieve data statistics after the call to .FinishDataLoad()
, you can easily use .GetStatistics()
to retrieve a data statistics class with information like:
Column names,
Types
Fill rates
Unpopulated row counts
Jagged counts
Row counts
This is a helpful method for post load data analysis.
Last updated