This is an old revision of the document!


Administration: Omnidex Indexing

Indexing Creation

Performance

Omnidex puts on premium on indexing performance. Applications often use hundreds of Omnidex indexes, and so it is important that all of these indexes can be built quickly. Omnidex also puts a premium on index size. Omnidex uses compression algorithms to make the indexes as small as possible while still allowing fast access. This allows the index to be quickly retrieved from the disk, and it also insures that memory usage is kept at a minimum. This improves performance by insuring the fastest I/O and insuring the most caching.

The help insure performance, administrators should maintain the Omnidex indexes on disk drives that are regularly defragmented, and have plenty of extra space. It is also valuable to isolate Omnidex indexes onto a separate disk drive, and even a separate disk controller, than the data files and the temporary file directory. In an ideal world, each of these three kinds of file would have their own drive and their own disk controller to minimize disk contention.

For large databases, administrators should also insure that indexes are not loaded at the same time queries are being performed. These two operations compete with each other such that both will go quite slowly. Administrators should either load indexes on a separate server, or should avoid queries while the indexes are being loaded.

Predicting Indexing Time

It is possible to get a rough estimate of the time required for indexing a table. This is only a rough estimate, as there are many variables. Chief among the variables are:

  • The speed of the server.
  • the load on the server

Other key variables are:

  • the distribution of the data, indexes and temporary directories on different disk drives and disk controllers
  • the number of indexes per table

(Number of rows) + (Number of prejoined rows) + (Number of keywords)


                  1 billion keywords per hour

Additional Resources

See also:

 
Back to top
admin/indexing/creation/performance.1327613837.txt.gz ยท Last modified: 2016/06/28 22:38 (external edit)