Integration: Raw Data Files

Fixed-length Files

Overview

In general, Omnidex data files must maintain a consistent structure, meaning that the data has consistent rows and columns. For example, most relational database systems allow data to be exported into data files, and these files are ideal for indexing with Omnidex. Similarly, companies often receive data from vendors or suppliers in this same form, and these files can be indexed directly without having to load the data into a relational database.

Fixed length files will always use the same number of bytes for each column and each row, regardless of the content of the data. No delimiters are used, and instead each column and each row can be located based on its offset within the file. Binary data such as integers, floating point, and date datatypes are stored in their native, binary format.

In the example below, each row consumes 44 bytes of the file, with the first row starting at the beginning of the file, the second row beginning at offset 44, the third row beginning at offset 88, and so forth. Note that the STRING datatype stores one more byte than the number of characters allowed, which is storage for the terminating NULL character. Also note that the FLOAT datatype requires 4 bytes regardless of the number of digits displayed, since a binary floating point value always requires 4 bytes of storage.

Column STATE DESCRIPTION STATE_CODE REGION COUNTRY TAX_RATE
Datatype CHAR(2) STRING(31) CHAR(2) CHAR(2) CHAR(2) FLOAT
Bytes of Storage 2 32 2 2 2 4
Offset 0 AK Alaska 02 PC US 0.000
Offset 44 AL Alabama 01 ES US 4.000
Offset 88 AR Arkansas 05 WS US 4.625

Additional Resources

See also:

 
Back to top
integration/rawdata/fixed/home.txt ยท Last modified: 2016/06/28 22:38 (external edit)