This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
admin:optimization:plans:example [2012/02/02 16:33] doc |
admin:optimization:plans:example [2016/06/28 22:38] (current) |
||
|---|---|---|---|
| Line 3: | Line 3: | ||
| {{page>:top_add&nofooter&noeditbtn}} | {{page>:top_add&nofooter&noeditbtn}} | ||
| - | ====== Administration ====== | + | ====== Administration: Optimizing Queries ====== |
| - | ===== Optimizing Queries ===== | + | ===== Query Plans ===== |
| - | **[[admin:optimization:plans:home|Overview]]** | | + | [[admin:optimization:plans:home|Overview]] | |
| [[admin:optimization:plans:reference|Anatomy of a Query Plan]] | | [[admin:optimization:plans:reference|Anatomy of a Query Plan]] | | ||
| - | [[admin:optimization:plans:example|Reading a Query Plan]] | | + | **[[admin:optimization:plans:example|Reading a Query Plan]]** | |
| [[admin:optimization:plans:configuration|Configuring Query Plans]] | | [[admin:optimization:plans:configuration|Configuring Query Plans]] | | ||
| [[admin:optimization:plans:optimization|Optimizing Queries]] | [[admin:optimization:plans:optimization|Optimizing Queries]] | ||
| Line 19: | Line 19: | ||
| ==== Reading a Query Plan ==== | ==== Reading a Query Plan ==== | ||
| - | The following is a rather straightforward query that joins two tables together --- Individuals and their respective Households --- and aggregates counts of all people born since 1980 that are either in Denver, CO or in Phoenix, AZ. The result is a display of counts by Gender. Let's take a look at how Omnidex optimizes this query. | + | The following is a rather straightforward query that joins two tables together --- Individuals and their respective Households --- and aggregates counts of all people born since 1980 that are either in Denver, CO or in Phoenix, AZ. The result is a display of counts by Gender. Let's take a look at the query plan to understand how Omnidex optimizes this query. |
| <html> | <html> | ||
| Line 26: | Line 26: | ||
| <td valign="top"> | <td valign="top"> | ||
| <font face="'Courier New'"> | <font face="'Courier New'"> | ||
| - |   Select   I.GENDER, count(*) <br /> | + |   Select      I.GENDER, count(*) <br /> |
| - |   from   HOUSEHOLDS H <br /> | + |   from      HOUSEHOLDS H <br /> |
| - |   join   INDIVIDUALS I <br /> | + |   join      INDIVIDUALS I <br /> |
| - |   on   H.HOUSEHOLD = I.HOUSEHOLD <br /> | + |     on    H.HOUSEHOLD = I.HOUSEHOLD <br /> |
| - |    where   ((H.STATE = 'CO' and <br /> | + |   where     ((H.STATE = 'CO' and <br /> |
| - |    H.CITY = 'Denver') or <br /> | + |   H.CITY = 'Denver') or <br /> |
| - |   (H.STATE = 'AZ' and <br /> | + |      (H.STATE = 'AZ' and <br /> |
| - |    H.CITY = 'Phoenix')) and <br /> | + |   H.CITY = 'Phoenix')) and <br /> |
| - |    I.BIRTHDATE > 'January 1, 1980' <br /> | + |   I.BIRTHDATE > 'January 1, 1980' <br /> |
| - |   group by   I.GENDER  | + |     group by  I.GENDER  |
| </font> | </font> | ||
| </td> | </td> | ||
| Line 58: | Line 58: | ||
| <td valign="top" width="50%"> | <td valign="top" width="50%"> | ||
| <font face="'Courier New'"> | <font face="'Courier New'"> | ||
| - |   Select   <font color=red>I.GENDER, count(*) </font><br /> | + |   Select      <font color=red>I.GENDER, count(*) </font><br /> |
| - |   <font color=red> from   HOUSEHOLDS H <br /> | + |   <font color=red> from      HOUSEHOLDS H <br /> |
| - |   join   INDIVIDUALS I <br /> | + |   join      INDIVIDUALS I <br /> |
| - |   on   H.HOUSEHOLD = I.HOUSEHOLD </font><br /> | + |     on    H.HOUSEHOLD = I.HOUSEHOLD </font><br /> |
| - |    where   ((H.STATE = 'CO' and <br /> | + |   where   ((H.STATE = 'CO' and <br /> |
| - |    H.CITY = 'Denver') or <br /> | + |   H.CITY = 'Denver') or <br /> |
| - |   (H.STATE = 'AZ' and <br /> | + |      (H.STATE = 'AZ' and <br /> |
| - |    H.CITY = 'Phoenix')) and <br /> | + |   H.CITY = 'Phoenix')) and <br /> |
| - |    I.BIRTHDATE > 'January 1, 1980' <br /> | + |   I.BIRTHDATE > 'January 1, 1980' <br /> |
| - |   <font color=red>group by   I.GENDER </font><br /> | + |     <font color=red>group by  I.GENDER </font><br /> |
| </br> | </br> | ||
| </font> | </font> | ||
| Line 73: | Line 73: | ||
| <td valign="top"> | <td valign="top"> | ||
| <font color=red> | <font color=red> | ||
| - | <-- <br /> | + | <--<br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| </font> | </font> | ||
| </td> | </td> | ||
| Line 125: | Line 124: | ||
| === Step 2: Process the HOUSEHOLDS table's criteria === | === Step 2: Process the HOUSEHOLDS table's criteria === | ||
| - | Omnidex will perform index qualifications to process the criteria in the HOUSEHOLDS table, paying attention to Boolean operators and parentheses to insure the correct result. Once this is done, Omnidex will have isolated the rows that meet this criteria, identified by a temporary file containing index pointers. | + | Omnidex will perform index qualifications to process the criteria in the HOUSEHOLDS table, paying attention to Boolean operators and parentheses to insure the correct result. Once this is done, Omnidex will have isolated index pointers for the rows that meet this criteria. |
| <html> | <html> | ||
| Line 139: | Line 138: | ||
| <td valign="top" width="50%"> | <td valign="top" width="50%"> | ||
| <font face="'Courier New'"> | <font face="'Courier New'"> | ||
| - |   Select   I.GENDER, count(*) <br /> | + |   Select      I.GENDER, count(*) <br /> |
| - |   from   HOUSEHOLDS H <br /> | + |   from      HOUSEHOLDS H <br /> |
| - |   join   INDIVIDUALS I <br /> | + |   join      INDIVIDUALS I <br /> |
| - |   on   H.HOUSEHOLD = I.HOUSEHOLD <br /> | + |     on    H.HOUSEHOLD = I.HOUSEHOLD <br /> |
| - |    where   <font color=red>((H.STATE = 'CO' and <br /> | + |   where   <font color=red>((H.STATE = 'CO' and <br /> |
| - |    H.CITY = 'Denver') or <br /> | + |   H.CITY = 'Denver') or <br /> |
| - |   (H.STATE = 'AZ' and <br /> | + |      (H.STATE = 'AZ' and <br /> |
| - |    H.CITY = 'Phoenix'))</font> and <br /> | + |   H.CITY = 'Phoenix'))</font> and <br /> |
| - |    I.BIRTHDATE > 'January 1, 1980' <br /> | + |   I.BIRTHDATE > 'January 1, 1980' <br /> |
| - |   group by   I.GENDER <br /> | + |     group by  I.GENDER <br /> |
| </br> | </br> | ||
| + | |||
| </font> | </font> | ||
| </td> | </td> | ||
| Line 207: | Line 207: | ||
| === Step 3: Join from HOUSEHOLDS to INDIVIDUALS === | === Step 3: Join from HOUSEHOLDS to INDIVIDUALS === | ||
| - | Omnidex will join from the HOUSEHOLDS table to the INDIVIDUALS table. Omnidex has several techniques for optimizing table joins. In this case, it will use the primary key values in HOUSEHOLDS as criteria against thed HOUSEHOLD index in the INDIVIDUALS table. Once this is done, Omnidex will have isolated rows in the INDIVIDUALS table that meet the criteria from the HOUSEHOLDS table. | + | Omnidex will join from the HOUSEHOLDS table to the INDIVIDUALS table. Omnidex has several techniques for optimizing table joins. In this case, it will use the primary key values in HOUSEHOLDS as criteria against thed HOUSEHOLD index in the INDIVIDUALS table. Omnidex will then have isolated index pointers for rows in the INDIVIDUALS table that link to the previously qualified HOUSEHOLDS rows. |
| <html> | <html> | ||
| Line 221: | Line 221: | ||
| <td valign="top" width="50%"> | <td valign="top" width="50%"> | ||
| <font face="'Courier New'"> | <font face="'Courier New'"> | ||
| - |   Select   I.GENDER, count(*) <br /> | + |   Select      I.GENDER, count(*) <br /> |
| - |   <font color=red>from   HOUSEHOLDS H <br /> | + |   <font color=red>from      HOUSEHOLDS H <br /> |
| - |   join   INDIVIDUALS I <br /> | + |   join      INDIVIDUALS I <br /> |
| - |   on   H.HOUSEHOLD = I.HOUSEHOLD </font> <br /> | + |     on    H.HOUSEHOLD = I.HOUSEHOLD </font> <br /> |
| - |    where   ((H.STATE = 'CO' and <br /> | + |   where   ((H.STATE = 'CO' and <br /> |
| - |    H.CITY = 'Denver') or <br /> | + |   H.CITY = 'Denver') or <br /> |
| - |   (H.STATE = 'AZ' and <br /> | + |      (H.STATE = 'AZ' and <br /> |
| - |    H.CITY = 'Phoenix')) and <br /> | + |   H.CITY = 'Phoenix')) and <br /> |
| - |    I.BIRTHDATE > 'January 1, 1980' <br /> | + |   I.BIRTHDATE > 'January 1, 1980' <br /> |
| - |   group by   I.GENDER  | + |     group by  I.GENDER <br /> |
| - | </br> | + | </br> |
| </font> | </font> | ||
| </td> | </td> | ||
| Line 289: | Line 290: | ||
| === Step 4: Process the INDIVIDUALS table's criteria === | === Step 4: Process the INDIVIDUALS table's criteria === | ||
| - | Omnidex will perform index qualifications to process the criteria in the INDIVIDUALS table. This will further refine the index pointers already identified by processing in the HOUSEHOLDS table. | + | Omnidex will perform index qualifications to process the criteria in the INDIVIDUALS table. This will further refine the index pointers so that they reflect criteria from both tables, as well as the table join itself. |
| <html> | <html> | ||
| - | <table> | + | <table width="100%"> |
| <tr> | <tr> | ||
| - | <td valign="top"> | + | <td colspan="3"> |
| <font face="'Courier New'"> | <font face="'Courier New'"> | ||
| - |   Select   I.GENDER, count(*) <br /> | + |   ----------------------------------- SUMMARY -----------------------------------</br> |
| - |     from   HOUSEHOLDS H <br /> | + | </font> |
| - |     join   INDIVIDUALS I <br /> | + | </td> |
| - |   on   H.HOUSEHOLD = I.HOUSEHOLD <br /> | + | </tr> |
| - |      where   ((H.STATE = 'CO' and <br /> | + | <tr> |
| - |   H.CITY = 'Denver') or <br /> | + | <td valign="top" width="50%"> |
| - |   (H.STATE = 'AZ' and <br /> | + | <font face="'Courier New'"> |
| - |   H.CITY = 'Phoenix')) and <br /> | + |   Select      I.GENDER, count(*) <br /> |
| - |   <font color=red>I.BIRTHDATE > 'January 1, 1980'</font> <br /> | + |   from      HOUSEHOLDS H <br /> |
| - |   group by   I.GENDER  | + |   join      INDIVIDUALS I <br /> |
| + |   on    H.HOUSEHOLD = I.HOUSEHOLD <br /> | ||
| + |   where   ((H.STATE = 'CO' and <br /> | ||
| + |   H.CITY = 'Denver') or <br /> | ||
| + |    (H.STATE = 'AZ' and <br /> | ||
| + |   H.CITY = 'Phoenix')) and <br /> | ||
| + |   <font color=red>I.BIRTHDATE > 'January 1, 1980' </font><br /> | ||
| + |   group by  I.GENDER <br /> | ||
| + | </br> | ||
| </font> | </font> | ||
| </td> | </td> | ||
| <td valign="top"> | <td valign="top"> | ||
| <font color=red> | <font color=red> | ||
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <-- <br /> | + | <--<br /> |
| </font> | </font> | ||
| </td> | </td> | ||
| <td valign="top"> | <td valign="top"> | ||
| <font color=red> | <font color=red> | ||
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | Omnidex will use the BIRTHDATE index to process the <br /> | + | Omnidex will use the BIRTHDATE index to <br /> |
| - | INDIVIDUALS table's criteria. | + | process INDIVIDUALS's criteria. |
| + | </font> | ||
| + | </td> | ||
| + | </tr> | ||
| + | <tr> | ||
| + | <td colspan="3"> | ||
| + | <font face="'Courier New'"> | ||
| + |   Version: 5.2.01 (Compiled Jan 23 2012 17:27:25) </br> | ||
| + |   ----------------------------------- DETAILS -----------------------------------</br> | ||
| + |   Qualify (HOUSEHOLDS)HOUSEHOLDS where CITY = 'Denver'; </br> | ||
| + |   Qualify (HOUSEHOLDS)HOUSEHOLDS where and STATE = 'CO'; </br> | ||
| + |   Create index segment O1 on 1; </br> | ||
| + |   Qualify (HOUSEHOLDS)HOUSEHOLDS where CITY = 'Phoenix'; </br> | ||
| + |   Qualify (HOUSEHOLDS)HOUSEHOLDS where and STATE = 'AZ'; </br> | ||
| + |   Qualify (HOUSEHOLDS)HOUSEHOLDS where or $ODXID = 'segment(O1)'; </br> | ||
| + |   Join HOUSEHOLDS using HOUSEHOLD to (INDIVIDUALS)INDIVIDUALS using HOUSEHOLD; </br> | ||
| + |  <font color=red> Qualify (INDIVIDUALS)INDIVIDUALS where and BIRTHDATE > '"January 1, 1980"'; </font></br> | ||
| + |   Aggregate INDIVIDUALS using GENDER for GROUP(GENDER), COUNT(*); </br> | ||
| + |   Return I.GENDER, COUNT('*'); </br> | ||
| + |   -------------------------------------------------------------------------------</br> | ||
| </font> | </font> | ||
| </td> | </td> | ||
| Line 342: | Line 370: | ||
| === Step 5: Aggregate counts in the INDIVIDUALS table=== | === Step 5: Aggregate counts in the INDIVIDUALS table=== | ||
| - | Omnidex will aggregate counts in the INDIVIDUALS table by scanning its indexes and filering the results using the index pointers set aside from the previous steps. | + | Omnidex will aggregate counts in the INDIVIDUALS table using indexes on the GROUP BY columns, while also intersecting with the index pointers that were isolated while processing criteria and table joins. |
| <html> | <html> | ||
| - | <table> | + | <table width="100%"> |
| <tr> | <tr> | ||
| + | <td colspan="3"> | ||
| + | <font face="'Courier New'"> | ||
| + |   ----------------------------------- SUMMARY -----------------------------------</br> | ||
| + | </font> | ||
| + | </td> | ||
| + | </tr> | ||
| + | <tr> | ||
| + | <td valign="top" width="50%"> | ||
| + | <font face="'Courier New'"> | ||
| + |   Select    <font color=red>I.GENDER, count(*) </font><br /> | ||
| + |   from    HOUSEHOLDS H <br /> | ||
| + |   join    INDIVIDUALS I <br /> | ||
| + |   on    H.HOUSEHOLD = I.HOUSEHOLD <br /> | ||
| + |   where   ((H.STATE = 'CO' and <br /> | ||
| + |   H.CITY = 'Denver') or <br /> | ||
| + |    (H.STATE = 'AZ' and <br /> | ||
| + |   H.CITY = 'Phoenix')) and <br /> | ||
| + |   I.BIRTHDATE > 'January 1, 1980' <br /> | ||
| + |   <font color=red>group by  I.GENDER </font><br /> | ||
| + | </br> | ||
| + | </font> | ||
| + | </td> | ||
| <td valign="top"> | <td valign="top"> | ||
| + | <font color=red> | ||
| + | <--<br /> | ||
| + | <br /> | ||
| + | <br /> | ||
| + | <br /> | ||
| + | <br /> | ||
| + | <br /> | ||
| + | <br /> | ||
| + | <br /> | ||
| + | <br /> | ||
| + | <-- </font> | ||
| + | </td> | ||
| + | <td valign="top"> | ||
| + | <font color=red> | ||
| + | Omnidex will aggregate counts using the <br /> | ||
| + | GENDER index, based on qualified rows. <br /> | ||
| + | <br /> | ||
| + | <br /> | ||
| + | <br /> | ||
| + | <br /> | ||
| + | <br /> | ||
| + | <br /> | ||
| + | <br /> | ||
| + | <br /> | ||
| + | </font> | ||
| + | </td> | ||
| + | </tr> | ||
| + | <tr> | ||
| + | <td colspan="3"> | ||
| <font face="'Courier New'"> | <font face="'Courier New'"> | ||
| - |   Select   <font color=red>I.GENDER, count(*) </font><br /> | + |   Version: 5.2.01 (Compiled Jan 23 2012 17:27:25) </br> |
| - |   from   HOUSEHOLDS H <br /> | + |   ----------------------------------- DETAILS -----------------------------------</br> |
| - |     join   INDIVIDUALS I <br /> | + |   Qualify (HOUSEHOLDS)HOUSEHOLDS where CITY = 'Denver'; </br> |
| - |   on   H.HOUSEHOLD = I.HOUSEHOLD <br /> | + |   Qualify (HOUSEHOLDS)HOUSEHOLDS where and STATE = 'CO'; </br> |
| - |      where   ((H.STATE = 'CO' and <br /> | + |   Create index segment O1 on 1; </br> |
| - |   H.CITY = 'Denver') or <br /> | + |   Qualify (HOUSEHOLDS)HOUSEHOLDS where CITY = 'Phoenix'; </br> |
| - |   (H.STATE = 'AZ' and <br /> | + |   Qualify (HOUSEHOLDS)HOUSEHOLDS where and STATE = 'AZ'; </br> |
| - |   H.CITY = 'Phoenix')) and <br /> | + |   Qualify (HOUSEHOLDS)HOUSEHOLDS where or $ODXID = 'segment(O1)'; </br> |
| - |   I.BIRTHDATE > 'January 1, 1980' <br /> | + |   Join HOUSEHOLDS using HOUSEHOLD to (INDIVIDUALS)INDIVIDUALS using HOUSEHOLD; </br> |
| - |   <font color=red>group by   I.GENDER </font> | + |   Qualify (INDIVIDUALS)INDIVIDUALS where and BIRTHDATE > '"January 1, 1980"'; </br> |
| + |  <font color=red> Aggregate INDIVIDUALS using GENDER for GROUP(GENDER), COUNT(*); </font></br> | ||
| + |   Return I.GENDER, COUNT('*'); </br> | ||
| + |   -------------------------------------------------------------------------------</br> | ||
| + | </font> | ||
| + | </td> | ||
| + | </tr> | ||
| + | </table> | ||
| + | </html> | ||
| + | |||
| + | === Step 6: Return the final result set === | ||
| + | |||
| + | Omnidex will finally return the result set of the query. When possible, Omnidex will return the first rows as soon as they are available, even if the later rows are still being processed. This speeds query execution, and also helps when only the first screenfuls of data are going to be viewed. | ||
| + | |||
| + | <html> | ||
| + | <table width="100%"> | ||
| + | <tr> | ||
| + | <td colspan="3"> | ||
| + | <font face="'Courier New'"> | ||
| + |   ----------------------------------- SUMMARY -----------------------------------</br> | ||
| + | </font> | ||
| + | </td> | ||
| + | </tr> | ||
| + | <tr> | ||
| + | <td valign="top" width="50%"> | ||
| + | <font face="'Courier New'"> | ||
| + |   Select    <font color=red>I.GENDER, count(*) </font><br /> | ||
| + |   from      HOUSEHOLDS H <br /> | ||
| + |   join      INDIVIDUALS I <br /> | ||
| + |   on    H.HOUSEHOLD = I.HOUSEHOLD <br /> | ||
| + |   where   ((H.STATE = 'CO' and <br /> | ||
| + |   H.CITY = 'Denver') or <br /> | ||
| + |    (H.STATE = 'AZ' and <br /> | ||
| + |   H.CITY = 'Phoenix')) and <br /> | ||
| + |   I.BIRTHDATE > 'January 1, 1980' <br /> | ||
| + |   group by  I.GENDER <br /> | ||
| + | </br> | ||
| </font> | </font> | ||
| </td> | </td> | ||
| <td valign="top"> | <td valign="top"> | ||
| <font color=red> | <font color=red> | ||
| - | <-- <br /> | + | <--<br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | + | ||
| </font> | </font> | ||
| </td> | </td> | ||
| <td valign="top"> | <td valign="top"> | ||
| <font color=red> | <font color=red> | ||
| - | Omnidex will use the GENDER index to aggregate counts <br /> | + | Omnidex will return the result set. <br /> |
| - | for the rows that have been isolated previously. <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | <br /> |
| - | <br /> | + | </font> |
| + | </td> | ||
| + | </tr> | ||
| + | <tr> | ||
| + | <td colspan="3"> | ||
| + | <font face="'Courier New'"> | ||
| + |   Version: 5.2.01 (Compiled Jan 23 2012 17:27:25) </br> | ||
| + |   ----------------------------------- DETAILS -----------------------------------</br> | ||
| + |   Qualify (HOUSEHOLDS)HOUSEHOLDS where CITY = 'Denver'; </br> | ||
| + |   Qualify (HOUSEHOLDS)HOUSEHOLDS where and STATE = 'CO'; </br> | ||
| + |   Create index segment O1 on 1; </br> | ||
| + |   Qualify (HOUSEHOLDS)HOUSEHOLDS where CITY = 'Phoenix'; </br> | ||
| + |   Qualify (HOUSEHOLDS)HOUSEHOLDS where and STATE = 'AZ'; </br> | ||
| + |   Qualify (HOUSEHOLDS)HOUSEHOLDS where or $ODXID = 'segment(O1)'; </br> | ||
| + |   Join HOUSEHOLDS using HOUSEHOLD to (INDIVIDUALS)INDIVIDUALS using HOUSEHOLD; </br> | ||
| + |   Qualify (INDIVIDUALS)INDIVIDUALS where and BIRTHDATE > '"January 1, 1980"'; </br> | ||
| + |   Aggregate INDIVIDUALS using GENDER for GROUP(GENDER), COUNT(*); </br> | ||
| + |  <font color=red> Return I.GENDER, COUNT('*'); </font></br> | ||
| + |   -------------------------------------------------------------------------------</br> | ||
| </font> | </font> | ||
| </td> | </td> | ||
| Line 403: | Line 535: | ||
| ====== Additional Resources ====== | ====== Additional Resources ====== | ||
| See also: | See also: | ||
| - | {{page>:admin:indexing:see_also&nofooter&noeditbtn}} | + | {{page>:admin:optimization:see_also&nofooter&noeditbtn}} |
| {{page>:bottom_add&nofooter&noeditbtn}} | {{page>:bottom_add&nofooter&noeditbtn}} | ||