New guidance on phpBB and system crons

A recent release of my digests extension revealed underlying problems not only in the extension, but also on guidance I’ve been giving for setting up phpBB and system crons. I can’t seem to edit the Wiki page, but I left notes on my digests extension discussion forum. I need to place them here for wider visibility. I will update the Wiki page when that technical issue is resolved.

With my digests extension, it’s generally important for digests to be mailed on time. Otherwise it depends on board traffic to send digests, which on sites with low traffic could result in significant delays in receiving digests.

There are two ways to do this in an automated way:

  • Create a system cron
  • Use a phpBB cron, but use a tool to call the board at least hourly, which kicks off phpBB’s cron process, which includes handling any scheduled outgoing digests

With a system cron, it turns out that you can’t use a semicolon to separate commands on one line in most cases. So this guidance is wrong in most cases. It all depends on the Linux shell used by the root user, which is usually bash. So when programming a cron job, to get two commands to work on one line, you need this instead:

cd /path/to/board && ./bin/phpbbcli.php cron:run

The && acts as a conditional, essentially saying that if the command to the left of the && succeeds, then issue the command to its right.

On my test board I also discovered I had to change the permissions on the /bin/phpbbcli.php program to 755, as the cron needs the execute permission, and it’s lacking with 644 permissions. This is not ideal as this introduces a potential security issue. Considering it’s just one file, I consider the security implications minor at best. With a system cron, you need to program a real cron job and tell phpBB to not use its built in cron based on board traffic: ACP > General > Server settings > Server settings > Run periodic tasks from system cron > Yes

If you want to use phpBB’s built in cron, you need to call it at least hourly and create a cron using curl, wget or lynx to hit your phpBB board as if it were a browser. If you follow the Wiki approach it causes a HTTP redirect, which basically causes it to fail. So the cron should look more like this (all on one line):

* * * * * curl -k -A='Mozilla/5.0'

The key here is to use app.php in the cron, to avoid the redirect.

Fixing phpBB emailing problems

Note: this post was updated on August 7, 2019.

Having trouble sending out emails such as mass emails or email notifications using phpBB? If so, you are not alone. Sending mass emails is a feature of phpBB (ACP > System > General tasks > Mass email). Users can also sign up for various email notifications in the User Control Panel. For some boards though these email notifications just doesn’t seem to work reliably, and sometimes don’t work at all. Perhaps some but not all of these emails go out.

Even if you don’t send out mass email regularly, phpBB has a number of features that depend on outgoing email servers to send out emails in a timely manner to all intended recipients. It’s been my experience that solving these issues is hard. This is particularly hard on shared hosting where there are quotas on outgoing emails. It’s made more confusing by phpBB’s inelegant interface for dealing with emails.

phpBB hands off email, generally to your host’s email server. Generally you have no idea where this email server is, what its policies are and how to determine if the email server is actually sending the emails. It often requires a support ticket to your web host, who may or may not provide accurate information.

Sorting this out can be very confusing which is why you might want to hire me. In most cases though I will have to work with your web host to fix these problems. Here are some useful techniques that can often solve these issues.

Setting phpBB’s email settings correctly

You can find your forum’s email settings in the Administration Control Panel: ACP > General > Client communications > Email settings. Things to check:

  • Enable board-wide emails. This should be Yes. Obviously if set to No, no emails will go out.
  • SMTP settings > Use SMTP server for email.
    • This is normally set to No. PHP is configured to send emails through your web host’s email server. This makes emailing simple in that you don’t need to know anything about the configuration of the email server. However, sometimes this will bite you. Here’s a list of reasons why using SMTP instead might be better approach.
    • If you have Windows hosting, you usually have to set this to Yes. In addition you will have to configure the SMTP server information fields appropriately using information provided by your web host. Getting SMTP email configured correctly is often challenging so SMTP, so if you are not sure what your SMTP settings should be, ask your web host. In addition, some web hosts don’t allow their servers to connect with external SMTP email servers. If you see in phpBB’s error log (ACP > Maintenance > Forum logs > Error log) messages like “Could not connect to smtp host : 110 : Connection timed out” this is probably because your host won’t allow external SMTP connections. If they provide an internal SMTP server, that is usually allowed.
  • Email package size. This is the maximum number of emails that will go out in one batch. So if you have 40 emails to send out and this is set to 20, the first 20 go out as a group. What happens to the other 20? They go into a queue. The next time there is traffic on your forum (or the next time a system cron is triggered, if you are set up to use a system cron), phpBB will attempt to send out the next 20 emails in the queue. Set this to 0 and the queue goes away. There are some problems with this approach:
    • It’s slower because it takes time to send out an email to one user, wait for an acknowledgement from the mail server, then send the next
    • It’s possible that this will take so long that the process will time out, something more likely on shared hosting
    • If you hit some sort of outgoing email quota, not all emails will go out and you may not be aware there is a problem
  • Contact email address and return email address. These email addresses should exist as real mailboxes. Your web host control panel should have a feature allowing you to create email accounts. (You can also set up forwarders to send any email these accounts receive to one you will read.) In addition, it is highly advisable that these email addresses use your domain. Why? It lessens the likelihood that the email will be considered spam because:
    • It’s coming from an email address associated with the IP address of your domain
    • Because these email addresses actually exist it suggests the email is legitimate
  • Hide email addresses. If the same email is going to multiple recipients, the email addresses will be BCCed (Blind Carbon Copied) instead of placed on the TO field. Unless your host has a policy of not allowing BCCed emails to be sent, you should leave this set to Yes.

Verify your site name

ACP > General > Board configuration > Board settings. Verify that the Site name field is filled in. Some administrators will blank this out because they don’t want it to appear on their style, as they have substituted a logo for their site name. This is a mistake because outgoing emails, such as email notifications are not distinct. This causes them to look like spam because many emails use a same generic subject, a trigger for many spam emails. These emails will often be blocked from being sent at all, or placed in a spam folder if they are received by the recipient. If necessary, you can edit the overall_header.html file in your style’s template folder to hide or remove the site name. (Purge the cache afterward for the change to take effect.)

Verify your web host’s email policies

Since most phpBB forums are on shared servers, it’s important to know what limits there are if any for sending out emails. This can often be found buried in the host’s knowledge base, but sometimes it takes a support ticket to know exactly what policies apply. Based on this information you can tune phpBB’s emailing algorithm to match your web host’s policies. Policies often have constraints like:

  • Only X emails can be sent over Y minutes or hours and/or
  • A maximum of Z distinct email addresses can be sent over Y minutes or hours and/or
  • Email FROM addresses must be real email addresses associated with your domain and/or
  • Sending emails using BCC is/is not allowed

Setting the correct email package size

Based on knowing your host’s email policies you can figure out a correct email package size. For example, if a maximum of 25 emails can be sent every 5 minutes, a good email package size would be 25, although it might be better to set it a bit lower, perhaps to 20. If 25 are allowed and you set it to 40, then it’s possible 25 will go out and 15 will not.

Using a system cron

If your email package size is greater than 0, then phpBB will create a queue of emails if necessary. So if you have 25 to go out but specify 20 in a package, 20 will go out and 5 will go into a queue. When do the next five go out? By default they will go out the next time there is web traffic to your forum. (ACP traffic doesn’t count.) So if your board doesn’t get much traffic, it could be hours or days later before the next 5 emails go out. This is not good if your users expect timely notifications.

A better way is to use a system cron. This helps send emails out on a timely basis plus it allows you to be congruent with your web host’s email policies. Creating a system cron is a bit tricky and may require someone with these skills to set it up. Of course you can hire me to do this for you. Procedures are outlined on phpBB’s Wiki.

To enable a system cron, go to ACP > General > Server configuration > Server settings. Set Run periodic tasks from system cron to Yes, then program the necessary cron job on your web server. This will disable phpBB’s default cron that depends on board traffic. Instead, depending on the interval you set with the cron job, a system cron built into your web server’s operating system will trigger periodic events that will process the email queue and other phpBB work.

In the above example, if we know the policy is not to permit more than 25 emails per five minutes, a cron job like this could be programmed to run every five minutes:

*/5 * * * * cd /path/to/board; php ./bin/phpbbcli.php cron:run

Using a pseudo-system cron

On shared hosting this often won’t work. This can be because:

  • You don’t know or can’t get the full path to your forum. A relative path won’t work. You need the full path from the server’s root folder. And what you see in tools like your web host control panel may show the full path, as the same machine is used by multiple clients.
  • Your cron tool won’t allow multiple commands. In the above example, the semicolon distinguishes one command from another.

You can still still program a cron however. In this case though you have the cron pretend to be a browser hitting your forum. See the Wiki for details, but something like this might work instead. (Note in this case you should not need to set Run periodic tasks from system cron to Yes. This way board traffic might cause email notifications to go out sooner, but if not the cron job will simulate a human hitting the board, which will do the job.)

*/5 * * * * curl -A=Mozilla/4.0

However, you might want to change a phpBB setting called queue_interval. This is normally set to 60, for 60 seconds. This means that if there is traffic on your forum a “phpBB cron” won’t be attempted unless at least 60 seconds has elapsed. The only case where you might want to change this is if you know your web host has an hourly or daily limit on outgoing emails. The Wiki talks about how to change the queue_interval.

Troubleshooting outgoing email issues

If there are errors sending out emails, phpBB may trap these in its Error log: ACP > Maintenance > Forum logs > Error log

Sometimes though phpBB cannot detect problems. You may find clues in your web server error log, if one exists and you can read it. Sometimes you will see an error_log file in your phpBB root folder, or in the website’s root folder, or in a /log directory that can be found with the web host’s File Manager. Looking at it or downloading it for examination may show error messages that will point to the root of the problem.

In many cases though you can’t solve them. You will have to get your web host involved. They will have to look at email server logs to see if anything shows there and provide guidance.

Sometimes it helps to look at your email queue, which is the file /cache/production/queue.php. This is a file that contain emails that haven’t been sent yet. If it doesn’t exist, it means there is nothing in the queue. If the queue.php file is very large, it may be that more recent emails are waiting for processing.

The queue.php file is actually a PHP program created by phpBB which is created and maintained dynamically. The email messages are wrapped inside of PHP <?php and ?> tags. If you want, you can carefully use a tool like your web host control panel’s file manager to view the file  The overall size of the file will give you an idea of how big the queue is. The dates of the emails can be seen if you carefully look at the file in the file manager. It helps to look at the top and bottom of the file to get a sense of what’s in the queue. Each email header contains an ISO date string like this:

Date: Mon, 05 Aug 2019 01:35:08 +0000

In this case, it indicates the email was placed into the queue on August 5, 2019 at around 0135 hours GMT.

The queue is processed from top to bottom. The number of emails sent from the queue when a phpBB cron event occurs should not exceed your email package size. The emails sent are removed from the queue. If there are new emails, they are added to the bottom of the file.

Solving emailing issues

If there are no log entries in phpBB’s error log indicating emailing issues, as far as phpBB is concerned the email was successfully sent. However, this may be wrong. If emails are still not being received, there are two likely reasons:

  • Your email server is blocking them from being sent, because they are being flagged as spam. It may not be all outgoing emails, but only certain emails that look like spam that are blocked. Figuring this out usually requires a telephone call with your web host. If you can provide examples of emails that did not go out with the approximate date and time sent, this will help them figure it out.
  • The recipient’s email server is deciding they are spam. Typically they are placed in a spam folder, but it could be they are simply trashed too. Ask users who are complaining to look at their spam folder. Most email programs allow users to create rules to automatically move certain emails in the spam folder into their inbox instead.

Getting off blacklists

There are lots of blacklists out there. These are lists of suspicious email servers that are reportedly sending out spam. This tool can tell you if your domain is blacklisted. If it’s all green, perhaps there is an issue with your web host’s email server.

If your domain is on one or more blacklists, you should try to get off them. It’s possible you are actually sending spam. For example, malware could have been installed on your server. If so it must be removed. Your web host may require you to scan your site for malware, or to assert there is none on it. They may then stop blocking outgoing email.

You can also appeal to each blacklist operator to remove your domain from its list.

A more effective, long-term strategy is to use Sender Policy Framework (SPF). This is basically a way that you assert “only these email servers are authorized to send email for my domain. Treat email coming from anywhere else as spam”. An SPF record is added to your domain’s DNS record. It looks something like this:

v=spf1 a mx ip4: ~all

You can check your domain’s current SPF record here.

To change your record, you may need to ask your web host for the IP of the server your domain uses to send email. You also need to assert any other SMTP servers you use for sending email from the domain. For example, if you use Outlook to reply to email sent to, the SMTP server configured in Outlook should be in this record too.