Drupal Performance: Pagination and COUNT queries

Drupal's core pagination system does not scale well on large, high-traffic sites because of it's need to count the total number of items in the list. This involves wrapping the original query around a COUNT query like below.

SELECT COUNT(*) FROM (SELECT nid FROM node);

Count queries are usually very fast for tables using the MyISAM engine, but most high-traffic Drupal sites use the InnoDB engine for most of the tables.

Learn to Develop Drupal Modules with the Examples Project

If you are looking to learn the Drupal API so you can develop your own modules, check out the Examples for Developers project. It includes highly commented example code covering some of the most common features of Drupal. Versions exist for both Drupal 6 and 7.

Website Performance Slides

I gave a presentation tonight to the Dayton Web Developers meetup on Website Performance. The presentation gave a high level view of both back end and front end performance considerations.

Drupal Performance Tuning

I work on several high traffic Drupal sites that get millions of page views per month. These sites require special fine tuning to make sure they stay online under heavy load and serve pages to users quickly.

Tuning a Drupal site for performance involves looking at several layers of the web application.

  1. The Web Server
  2. The PHP code
  3. The Database and queries
  4. HTML Components
  5. Javascript

Items 1 to 3 are in the category of Back End Performance and involve delivering the main HTML document to the user's browser as quickly as possible.

Drupal performance and the variable cache

Lately, several of the Drupal sites I work on with large amounts of traffic have been showing many sluggish queries. One query that kept constantly showing up was this one:

SELECT * FROM variable;

Drupal stores most of it's settings in the variable table, but to improve query performance it stores all the variables from this table into a single record inside the cache table. However, when a variable is changed, added or deleted that cache must be updated.

The reason the above query was getting executed so frequently was because of a variable_set() call inside the Views module that was run on every page load.

I've seen this mistake more than once, so keep in mind that you should only add, change or delete variables during administrative tasks.

Drupal and Security

Last week at the first Dayton Drupal User Group I presented on Drupal and Security. Here are the slides from the presentation. Many thanks to Greg Knaddison for his book Cracking Drupal which was extremely helpful in putting together this presentation.

Organizing your theme files in Drupal

When working on a large Drupal site, your theme can become quite complicated with dozens of templates or template overrides, images, css files, javascript files and even a massively large template.php file. Sometimes all these files will get dumped into the root of the folder making specific files difficult to find.

Here are some best practices when organizing your theme folder to make things easier to find.

CSS, Javascript and Images

Keep these files in separate folders like so:

theme-name/js/
theme-name/css/
theme-name/images/

or you may want to organize them into an assets folder like this:

theme-name/assets/js/
theme-name/assets/css/
theme-name/assets/images/

Template Files

Put all your template files into a templates directory, then further separate them by the base template type (ie. page, node, block, view).

Security and the global user object

I'm starting to go through the book Cracking Drupal by Greg Knaddison. The book is written primarily for developers and seems to cover a broad range of security topics specific to Drupal and lots of topics that apply to any web-based application.

Over the next week, I'll post some of the best gems I find in the book.

In this post, I'll share an issue I've seen a few times with the global user object.

When a user is logged into a site (or even if they are anonymous), their user information is stored in a global PHP variable called $user. If you're familiar with Drupal code at all, you've probably seen the global user object called in many functions, usually something like this:

<?php function some_function() { global $user; // Do something } ?>

As a global variable, the $user object can be easily accessed by any function.

RPX Module Features

RPX is a product from JanRain that enables websites to allow their users to registers or login using their accounts from popular 3rd party social services like Facebook, Twitter and Yahoo.

I began working on the RPX Module last summer in conjunction with some of the folks at JanRain who run the RPX service. Since that time we've made substantial progress, releasing the first full version in January.

Current Features

  • Easy RPX Registration: In order to the RPX service with your site, you need to register an account on rpxnow.com and setup an API key for your site.

Drupal Security Review

Drupal is a very secure open source project. With thousands of eyes looking at the code, security holes get patched very quickly in the core Drupal project.

However, one of the primary ways of opening up your Drupal site to security vulnerabilities is through user error. No amount of code can prevent site administrators from doing stupid things like giving anonymous users permission to edit your blog posts.