Integration: Raw Data Files

Delimited Files

Offset Indexes

Delimited files have variable length rows, and this makes it difficult to locate a particular row. The offset for a particular row in a fixed-length file is easy to calculate by multiplying the record length times the row number. Delimited files require a different approach. Omnidex uses an Offset Index (OFX) to record the unique offsets for each row.

An Offset Index is a file maintained in the same location as the data file that contains an array of unique offsets for each row. The offset for the first row is located in the first eight bytes of the file; the offset for the second row is located in the next eight bytes of the file; and so on. This file is accessed whenever a row number needs to be translated into a physical offset of the file, and it is required if a delimited file will be indexed with Omnidex.

This file must be regenerated when the delimited file initially deployed. To create an Offset Index, connect to the environment and issue the LOAD OFX statement:

> load ofx for table states
Loaded ofx for STATES with 76 entries

Offset Indexes are maintained automatically when Omnidex INSERT statements are issued. If the file is replaced or altered outside of the INSERT statement, then the LOAD OFX statement should be reissued to regenerate the Offset Index.

Additional Resources

See also:

Back to top
integration/rawdata/delimited/ofx.txt ยท Last modified: 2016/06/28 22:38 (external edit)