Building an IMAP Email Client with PHP

Developers sometimes run into tasks that require access to email mailboxes. In most cases, this is done using the Internet Message Access Protocol, or IMAP. As a PHP developer, I first turned to PHP’s built in IMAP library, but this library is buggy and impossible to debug or modify. It is also not possible to customize IMAP commands to make full use of the protocol’s abilities.

So today, we will create a working IMAP email client from the ground up using PHP. We will also see how to use Gmail’s special commands.

We will implement IMAP in a custom class, imap_driver. I will explain each step while building the class. You can download the whole imap_driver.php at the end of the article.

Establishing a Connection

IMAP is a connection-based protocol and typically operates over TCP/IP with SSL security, so before we can make any IMAP calls we must open the connection.

We need to know the URL and port number of the IMAP server we want to connect to. This information is usually advertised in the service’s website or documentation. For example, for Gmail, the URL is ssl://imap.gmail.com on port 993.

Since we want to know if initialization was successful, we will leave our class constructor empty, and all the connections will be made in a custom init() method, which will return false if the connection cannot be established:

class imap_driver
{
    private $fp;                      // file pointer
    public $error;                    // error message
    ...
    public function init($host, $port)
    {
        if (!($this->fp = fsockopen($host, $port, $errno, $errstr, 15))) {
            $this->error = "Could not connect to host ($errno) $errstr";
            return false;
        }
        if (!stream_set_timeout($this->fp, 15)) {
            $this->error = "Could not set timeout";
            return false;
        }
        $line = fgets($this->fp);     // discard the first line of the stream
        return true;
    }
    
    private function close()
    {
        fclose($this->fp);
    }
    ...
}

In the above code, I’ve set a timeout of 15 seconds, both for fsockopen() to establish the connection, and for the data stream itself to respond to requests once it is open. It is important to have a timeout for every call to the network because, often enough, the server won’t respond, and we must be able to handle such a freeze.

I also grab the first line of the stream and ignore it. Usually this just a greeting message from the server, or a confirmation that it is connected. Check your particular mail service’s documentation to make sure this is the case.

Now we want to run the above code to see that init() is successful:

include("imap_driver.php");

// test for init()
$imap_driver = new imap_driver();
if ($imap_driver->init('ssl://imap.gmail.com', 993) === false) {
    echo "init() failed: " . $imap_driver->error . "\n";
    exit;
}

Basic IMAP Syntax

Now that we have an active socket open to our IMAP server, we can start sending IMAP commands. Let us take a look at IMAP syntax.

The formal documentation can be found in Internet Engineering Task Force (IETF) RFC3501. IMAP interactions typically consist of the client sending commands, and the server responding with an indication of success, along with whatever data may have been requested.

The basic syntax for commands is:

line_number command arg1 arg2 ...

The line number, or “tag”, is a unique identifier for the command, that the server uses to indicate which command it is responding to should it be processing multiple commands at once.

Here is an example, showing the LOGIN command:

00000001 LOGIN example@gmail.com password

The server’s response may begin with an “untagged” data response. For instance, Gmail responds to a successful login with an untagged response containing information about the server’s capabilities and options, and a command to fetch an email message will receive an untagged response containing the message body. In either case, a response should always end with a “tagged” command completion response line, identifying the line number of the command that the response applies to, a completion status indicator, and additional metadata about the command, if any:

line_number status metadata1 metadata2 ...

Here is how Gmail responds to the LOGIN command:

• Success:

* CAPABILITY IMAP4rev1 UNSELECT IDLE NAMESPACE QUOTA ID XLIST CHILDREN X-GM-EXT-1 UIDPLUS 
COMPRESS=DEFLATE ENABLE MOVE CONDSTORE ESEARCH UTF8=ACCEPT LIST-EXTENDED LIST-STATUS

00000001 OK example@gmail.com authenticated (Success)

• Failure:

00000001 NO [AUTHENTICATIONFAILED] Invalid credentials (Failure)

The status can be either OK, indicating success, NO, indicating failure, or BAD, indicating an invalid command or bad syntax.

Implementing Basic Commands:

Let’s make a function to send a command to the IMAP server, and retrieve the response and endline:

class imap_driver
{
    private $command_counter = "00000001";
    public $last_response = array();
    public $last_endline = "";

    private function command($command)
    {
        $this->last_response = array();
        $this->last_endline  = "";
        
        fwrite($this->fp, "$this->command_counter $command\r\n");            // send the command
        
        while ($line = fgets($this->fp)) {                                   // fetch the response one line at a time
            $line = trim($line);                                             // trim the response
            $line_arr = preg_split('/\s+/', $line, 0, PREG_SPLIT_NO_EMPTY);  // split the response into non-empty pieces by whitespace
            
            if (count($line_arr) > 0) {
                $code = array_shift($line_arr);                              // take the first segment from the response, which will be the line number
                
                if (strtoupper($code) == $this->command_counter) {
                    $this->last_endline = join(' ', $line_arr);              // save the completion response line to parse later
                    break;
                } else {
                    $this->last_response[] = $line;                          // append the current line to the saved response
                }
                
            } else {
                $this->last_response[] = $line;
            }
        }
        
        $this->increment_counter();
    }

    private function increment_counter()
    {
        $this->command_counter = sprintf('%08d', intval($this->command_counter) + 1);
    }
    ...
}

The LOGIN Command

Now we can write functions for specific commands that call our command() function under the hood. Let’s write a function for the LOGIN command:

class imap_driver
{ 
    ...
    public function login($login, $pwd)
    {
        $this->command("LOGIN $login $pwd");
        if (preg_match('~^OK~', $this->last_endline)) {
            return true;
        } else {
            $this->error = join(', ', $this->last_response);
            $this->close();
            return false;
        }
    }
    ...
}

Now we can test it like this. (Note that you must have an active email account to test against.)

...
// test for login()
if ($imap_driver->login('example@gmail.com', 'password') === false) {
    echo "login() failed: " . $imap_driver->error . "\n";
    exit;
}

Note that Gmail is very strict about security by default: it will not allow us to access an email account with IMAP if we have default settings and try to access it from a country other than the account profile’s country. But it is easy enough to fix; just set less secure settings in your Gmail account, as described here.

The SELECT Command

Now let’s see how to select an IMAP folder in order to do something useful with our email. The syntax is similar to that of LOGIN, thanks to our command() method. We use the SELECT command instead, and specify the folder.

class imap_driver
{
    ...
    public function select_folder($folder)
    {
        $this->command("SELECT $folder");
        if (preg_match('~^OK~', $this->last_endline)) {
            return true;
        } else {
            $this->error = join(', ', $this->last_response);
            $this->close();
            return false;
        }
    }
    ...
}

To test it, let’s try to select the INBOX:

...
// test for select_folder()
if ($imap_driver->select_folder("INBOX") === false) {
    echo "select_folder() failed: " . $imap_driver->error . "\n";
    return false;
}

Implementing Advanced Commands

Let’s look at how to implement a few of IMAP’s more advanced commands.

The SEARCH Command

A common routine in email analysis is to search for emails in a given date range, or search for flagged emails, and so on. The search criteria must be passed to the SEARCH command as an argument, with space as a separator. For example, if we want to get all emails since November 20th, 2015, we must pass the following command:

00000005 SEARCH SINCE 20-Nov-2015

And the response will be something like this:

* SEARCH 881 882
00000005 OK SEARCH completed

Detailed documentation of possible search terms can be found here The output of a SEARCH command is a list of UIDs of emails, separated by whitespace. A UID is a unique identifier of an email in the user’s account, in chronological order, where 1 is the oldest email. To implement the SEARCH command we must simply return the resulting UIDs:

class imap_driver
{
    ...
    public function get_uids_by_search($criteria)
    {
        $this->command("SEARCH $criteria");
        
        if (preg_match('~^OK~', $this->last_endline)
        && is_array($this->last_response)
        && count($this->last_response) == 1) {
        
            $splitted_response = explode(' ', $this->last_response[0]);
            $uids              = array();
            
            foreach ($splitted_response as $item) {
                if (preg_match('~^\d+$~', $item)) {
                    $uids[] = $item;                        // put the returned UIDs into an array
                }
            }
            return $uids;
            
        } else {
            $this->error = join(', ', $this->last_response);
            $this->close();
            return false;
        }
    }
    ...
}

To test this command, we will get emails from the last three days:

...
// test for get_uids_by_search()
$ids = $imap_driver->get_uids_by_search('SINCE ' . date('j-M-Y', time() - 60 * 60 * 24 * 3));
if ($ids === false)
{
    echo "get_uids_failed: " . $imap_driver->error . "\n";
    exit;
}

The FETCH Command with BODY.PEEK

Another common task is to get email headers without marking an email as SEEN. From the IMAP manual, the command for retrieving all or part of an email is FETCH. The first argument indicates which part we are interested in, and typically BODY is passed, which will return the entire message along with its headers, and mark it as SEEN. The alternative argument BODY.PEEK will do the same thing, without marking the message as SEEN.

IMAP syntax requires our request to also specify, in square brackets, the section of the email that we want to fetch, which in this example is [HEADER]. As a result, our command will look like this:

00000006 FETCH 2 BODY.PEEK[HEADER]

And we will expect a response that looks like this:

* 2 FETCH (BODY[HEADER] {438}
MIME-Version: 1.0
x-no-auto-attachment: 1
Received: by 10.170.97.214; Fri, 30 May 2014 09:13:45 -0700 (PDT)
Date: Fri, 30 May 2014 09:13:45 -0700
Message-ID: <CACYy8gU+UFFukbE0Cih8kYRENMXcx1DTVhvg3TBbJ52D8OF6nQ@mail.gmail.com>
Subject: The best of Gmail, wherever you are
From: Gmail Team <mail-noreply@google.com>
To: Example Test <example@gmail.com>
Content-Type: multipart/alternative; boundary=001a1139e3966e26ed04faa054f4
)
00000006 OK Success

In order to build a function for fetching headers, we need to be able to return the response in a hash structure (key/value pairs):

class imap_driver
{ 
    ...
    public function get_headers_from_uid($uid)
    {
        $this->command("FETCH $uid BODY.PEEK[HEADER]");
        
        if (preg_match('~^OK~', $this->last_endline)) {
            array_shift($this->last_response);                  // skip the first line
            $headers    = array();
            $prev_match = '';
            
            foreach ($this->last_response as $item) {
                if (preg_match('~^([a-z][a-z0-9-_]+):~is', $item, $match)) {
                    $header_name           = strtolower($match[1]);
                    $prev_match            = $header_name;
                    $headers[$header_name] = trim(substr($item, strlen($header_name) + 1));
                    
                } else {
                    $headers[$prev_match] .= " " . $item;
                }
            }
            return $headers;
            
        } else {
            $this->error = join(', ', $this->last_response);
            $this->close();
            return false;
        }
    } 
    ...
}

And to test this code we just specify the UID of the message we are interested in:

...
// test for get_headers_by_uid
if (($headers = $imap_driver->get_headers_from_uid(2)) === false) {
    echo "get_headers_by_uid() failed: " . $imap_driver->error . "\n";
    return false;
}

Gmail IMAP Extensions

Gmail provides a list of special commands that can make our life much easier. The list of Gmail’s IMAP extension commands is available here. Let’s review a command that, in my opinion, is the most important one: X-GM-RAW. It allows us to use Gmail search syntax with IMAP. For example, we can search for emails that are in the categories Primary, Social, Promotions, Updates, or Forums.

Functionally, X-GM-RAW is an extension of the SEARCH command, so we can reuse the code that we have above for the SEARCH command. All we need to do is add the keyword X-GM-RAW and criteria:

...
// test for gmail extended search functionality
$ids = $imap_driver->get_uids_by_search(' X-GM-RAW "category:primary"');
if ($ids === false) {
    echo "get_uids_failed: " . $imap_driver->error . "\n";
    return false;
}

The above code will return all UIDs that are listed in the “Primary” category.

Note: As of December 2015, Gmail often confuses the “Primary” category with the “Updates” category on some accounts. This is a Gmail bug that hasn’t been fixed yet.

Conclusion

Overall, the custom socket approach provides more freedom to the developer. It makes it possible to implement all commands in IMAP RFC3501. It will also give you better control over your code, since you don’t have to wonder what’s happening “behind the scenes.”

The full imap_driver class that we implemented in this article can be found here. It can be used as-is, and it will take only a few minutes for a developer to write a new function or request to their IMAP server. I’ve also included a debug feature in the class for a verbose output.