This shows you the differences between two versions of the page.
|
admin:optimization:plans:home [2012/02/01 23:47] doc |
admin:optimization:plans:home [2016/06/28 22:38] |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ~~NOTOC~~ | ||
| - | {{page>:top_add&nofooter&noeditbtn}} | ||
| - | |||
| - | ====== Administration ====== | ||
| - | |||
| - | ===== Optimizing Queries ===== | ||
| - | |||
| - | |||
| - | **[[admin:optimization:plans:home|Overview]]** | | ||
| - | [[admin:optimization:plans:sections|Sections]] | | ||
| - | [[admin:optimization:plans:steps|Steps]] | | ||
| - | [[admin:optimization:plans:optimized|Fully Optimized Queries]] | ||
| - | |||
| - | ---- | ||
| - | \\ | ||
| - | |||
| - | ==== Overview ==== | ||
| - | |||
| - | Omnidex always optimizes a query as well as it can using the Omnidex indexes; however, if the indexes are not enough, Omnidex will complete the query without indexes, insuring the correct result. In fact, Omnidex can process queries even when no Omnidex indexes are available at all. In this way, Omnidex is first and foremost a SQL Engine for both relational and non-relational, or NoSQL, databases. | ||
| - | |||
| - | Omnidex will evaluate the query and identify where indexes can be used. Omnidex evaluates the tables and their join relationships. Omnidex evaluates criteria, including nested queries, SQL functions and complex Boolean operations. Omnidex evaluates group by and order by, both to perform aggregations and to avoid unnecessary sorting of data. Omnidex even considers whether indexes can be used to return columns in the result set, avoiding accessing the data whenever possible. If there are not indexes to satisfy any of these steps, it will process then without the aid of indexing. | ||
| - | |||
| - | The optimization plan for a query shows a sequence of steps, including table joins, processing criteria, aggregating data, and retrieving from the database. The ideal with Omnidex optimization is to avoid retrieving from the database if possible, and to fully optimize the query solely through the Omnidex indexes. If non-indexed steps are required, optimization tries to minimize these steps as much as possible. | ||
| - | |||
| - | ==== Optimizing a Query ==== | ||
| - | |||
| - | The following is a rather straightforward query that joins two tables together --- Individuals and their respective Households --- and aggregates counts of all people born since 1980 that are either in Denver, CO or in Phoenix, AZ. The result is a display of counts by Gender. Let's take a look at how Omnidex optimizes this query. | ||
| - | |||
| - | <html> | ||
| - | <table> | ||
| - | <tr> | ||
| - | <td valign="top"> | ||
| - | <font face="'Courier New'"> | ||
| - |   Select   I.GENDER, count(*) <br /> | ||
| - |   from   HOUSEHOLDS H <br /> | ||
| - |   join   INDIVIDUALS I <br /> | ||
| - |   on   H.HOUSEHOLD = I.HOUSEHOLD <br /> | ||
| - |    where   ((H.STATE = 'CO' and <br /> | ||
| - |    H.CITY = 'Denver') or <br /> | ||
| - |   (H.STATE = 'AZ' and <br /> | ||
| - |    H.CITY = 'Phoenix')) and <br /> | ||
| - |    I.BIRTHDATE > 'January 1, 1980' <br /> | ||
| - |   group by   I.GENDER  | ||
| - | </font> | ||
| - | </td> | ||
| - | </tr> | ||
| - | </table> | ||
| - | </html> | ||
| - | |||
| - | === Step 1: Determine the processing order === | ||
| - | |||
| - | Omnidex must first determine the best processing order for the query. This is determined based on analyzing the table relationships and the select items. Generally speaking, Omnidex needs to end up in the table where the select items or aggregations reside, and must process the other tables first. In this example, Omnidex will recognize that the HOUSEHOLDS table is the parent table, and the query will aggregate data in the INDIVIDUALS table. It will process the HOUSEHOLDS criteria first. | ||
| - | |||
| - | <html> | ||
| - | <table> | ||
| - | <tr> | ||
| - | <td valign="top"> | ||
| - | <font face="'Courier New'"> | ||
| - |   Select   <font color=red>I.GENDER, count(*) </font><br /> | ||
| - |   from   <font color=red>HOUSEHOLDS H <br /> | ||
| - |   join   INDIVIDUALS I <br /> | ||
| - |   on   H.HOUSEHOLD = I.HOUSEHOLD </font><br /> | ||
| - |    where   ((H.STATE = 'CO' and <br /> | ||
| - |    H.CITY = 'Denver') or <br /> | ||
| - |   (H.STATE = 'AZ' and <br /> | ||
| - |    H.CITY = 'Phoenix')) and <br /> | ||
| - |    I.BIRTHDATE > 'January 1, 1980' <br /> | ||
| - |   <font color=red>group by   I.GENDER </font> | ||
| - | </font> | ||
| - | </td> | ||
| - | <td valign="top"> | ||
| - | <font color=red> | ||
| - | <-- <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | |||
| - | </font> | ||
| - | </td> | ||
| - | <td valign="top"> | ||
| - | <font color=red> | ||
| - | Since this query will aggregate the INDIVIDUALS table, <br /> | ||
| - | Omnidex will need to process the HOUSEHOLDS first <br /> | ||
| - | and then end in the INDIVIDUALS table. <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | |||
| - | </font> | ||
| - | </td> | ||
| - | </tr> | ||
| - | </table> | ||
| - | </html> | ||
| - | |||
| - | |||
| - | === Step 2: Process the HOUSEHOLD table's criteria === | ||
| - | |||
| - | Omnidex will perform index qualifications to process the criteria in the HOUSEHOLDS table. If will do pay attention to Boolean operators and parentheses to insure the correct result. Once this is done, Omnidex will have isolated the rows that meet this criteria, identified by a temporary file containing index pointers. | ||
| - | |||
| - | <html> | ||
| - | <table> | ||
| - | <tr> | ||
| - | <td valign="top"> | ||
| - | <font face="'Courier New'"> | ||
| - |   Select   I.GENDER, count(*) <br /> | ||
| - |   from   HOUSEHOLDS H <br /> | ||
| - |   join   INDIVIDUALS I <br /> | ||
| - |   on   H.HOUSEHOLD = I.HOUSEHOLD <br /> | ||
| - |    where   <font color=red>((H.STATE = 'CO' and <br /> | ||
| - |    H.CITY = 'Denver') or <br /> | ||
| - |   (H.STATE = 'AZ' and <br /> | ||
| - |    H.CITY = 'Phoenix'))</font> and <br /> | ||
| - |    I.BIRTHDATE > 'January 1, 1980' <br /> | ||
| - |   group by   I.GENDER  | ||
| - | </font> | ||
| - | </td> | ||
| - | <td valign="top"> | ||
| - | <font color=red> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | <-- <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | |||
| - | </font> | ||
| - | </td> | ||
| - | <td valign="top"> | ||
| - | <font color=red> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | Omnidex will use the STATE and CITY indexes to process <br /> | ||
| - | the HOUSEHOLDS table's criteria, paying attention to <br /> | ||
| - | Boolean operators and parentheses. <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | |||
| - | </font> | ||
| - | </td> | ||
| - | </tr> | ||
| - | </table> | ||
| - | </html> | ||
| - | |||
| - | |||
| - | === Step 3: Join from HOUSEHOLDS to INDIVIDUAL === | ||
| - | |||
| - | Omnidex will join from the HOUSEHOLDS table to the INDIVIDUALS table. Omnidex has several techniques for optimizing table joins. In this case, it will use the primary key values in HOUSEHOLDS as criteria against thed HOUSEHOLD index in the INDIVIDUALS table. Once this is done, Omnidex will have isolated rows in the INDIVIDUALS table that meet the criteria from the HOUSEHOLDS table. | ||
| - | |||
| - | <html> | ||
| - | <table> | ||
| - | <tr> | ||
| - | <td valign="top"> | ||
| - | <font face="'Courier New'"> | ||
| - |   Select   I.GENDER, count(*) <br /> | ||
| - |   <font color=red>from   HOUSEHOLDS H <br /> | ||
| - |   join   INDIVIDUALS I <br /> | ||
| - |   on   H.HOUSEHOLD = I.HOUSEHOLD </font> <br /> | ||
| - |    where   ((H.STATE = 'CO' and <br /> | ||
| - |    H.CITY = 'Denver') or <br /> | ||
| - |   (H.STATE = 'AZ' and <br /> | ||
| - |    H.CITY = 'Phoenix')) and <br /> | ||
| - |    I.BIRTHDATE > 'January 1, 1980' <br /> | ||
| - |   group by   I.GENDER  | ||
| - | </font> | ||
| - | </td> | ||
| - | <td valign="top"> | ||
| - | <font color=red> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | <-- <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | |||
| - | </font> | ||
| - | </td> | ||
| - | <td valign="top"> | ||
| - | <font color=red> | ||
| - | <br /> | ||
| - | Omnidex will use the primary keys from the HOUSEHOLDS table <br /> | ||
| - | as criteria against the HOUSEHOLD index in the INDIVIDUALS <br /> | ||
| - | table to process this table join. <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | <br /> | ||
| - | |||
| - | </font> | ||
| - | </td> | ||
| - | </tr> | ||
| - | </table> | ||
| - | </html> | ||
| - | |||
| - | |||
| - | |||
| - | [[admin:optimization:plans:home| More >]] | ||
| - | |||
| - | ===== ===== | ||
| - | ---- | ||
| - | \\ | ||
| - | **[[admin:optimization:overview:hdc|Next]]** | ||
| - | |||
| - | |||
| - | ====== Additional Resources ====== | ||
| - | See also: | ||
| - | {{page>:admin:indexing:see_also&nofooter&noeditbtn}} | ||
| - | |||
| - | {{page>:bottom_add&nofooter&noeditbtn}} | ||