Drupal Performance: Pagination and COUNT queries

Drupal's core pagination system does not scale well on large, high-traffic sites because of it's need to count the total number of items in the list. This involves wrapping the original query around a COUNT query like below.

SELECT COUNT(*) FROM (SELECT nid FROM node);

Count queries are usually very fast for tables using the MyISAM engine, but most high-traffic Drupal sites use the InnoDB engine for most of the tables. COUNT queries on tables using the InnoDB engine are notoriously slow and become painfully so the more records are stored in that table (think 100K+ nodes or comment).

I've seen one work-a-round for this from Robert Douglass on Aquia's blog, but, in my specific situation the issue involved Views and I needed a way to automatically use a pager that did not require these expensive, and sometimes crippling, COUNT queries.

As a solution, I wrote a simple module to integrate with Views that adds a new "Lite" option when adding a pager to a view. This "Lite" pager allows users to navigate between pages of content, but without counting the number of total results available. This new pager is only slightly less useful than Drupal's core pager in that users can't see the total number of pages of content or navigate to the last page, but I think the trade off in performance gains is well worth while.

If you are interested in using the Views Litepager module, checkout the project page on this site. If there is enough interest, I can add it as an official contrib module on Drupal.org.

Comments

The count(*) should be

The count(*) should be count(1) in MySQL. Significant performance increase in some cases.

Drupal rocks.

Drupal rocks.

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Allowed HTML tags: <a> <em> <strong> <strike> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • You can enable syntax highlighting of source code with the following tags: <code>, <blockcode>, <bash>, <css>, <drupal5>, <drupal6>, <javascript>, <mysql>, <php>, <python>, <ruby>. The supported tag styles are: <foo>, [foo].
  • You can use Markdown syntax to format and style the text. Also see Markdown Extra for tables, footnotes, and more.
  • Web page addresses and e-mail addresses turn into links automatically.

More information about formatting options

Type the characters you see in this picture. (verify using audio)
Type the characters you see in the picture above; if you can't read them, submit the form and a new image will be generated. Not case sensitive.