Digests 3.0.4 (RC3) Released

A list of file changes compared with 3.0.3 can be seen here. Among the changes:

  • Updated language files for German, thanks to Praggle
  • Since all digests are sent out when created instead of put in a queue, removed language variable DIGESTS_CREATED_FOR
  • Fixed post time and private message time to properly show these times in the user’s timezone based on the timezone shown in their board preferences
  • Fixed issue where more than one user can get the same digest for the same hour
  • composer.json has proper instructions for checking version

This is probably ready for a phpBB extension team review, but I’ll give it a few days to see if other issues are found. You can leave comments on the phpbb.com digests topic.

I think this is the first release that can be installed on a production system. There may be minor problems but it should work reliably if you follow the installation and testing instructions carefully.

Why aren’t posts appearing in my digests?

I get a lot of question on phpbb.com on why digests don’t contain expected posts. Some were definitely due to bugs. While developing the digest extension I recently closed a bug that had shifted the time window for a daily or weekly digest from being based on the user’s timezone to GMT. Downloading and installing the latest master from GitHub is usually a good idea if you are encountering bugs like this.

Digests is built around users having the ability to filter posts. So pretty much any option you select in the user control panel for digests allows you to constrain what appears in a digest.

Here are some of the many reasons why you may not be seeing posts in digests:

  • Your digest type is incorrect. If you wanted a weekly digest but selected a daily digest, you are only going to get posts for the last day. If you selected “None” (no digest), obviously nothing will be sent.
  • Your timezone is wrong. If your timezone in the user control panel (look under Board preferences) is GMT and your timezone is GMT -4 you may not see four hours of posts.
  • You selected first posts of new topics only. In this case you won’t see topic replies.
  • You selected bookmarked topics only. Either turn this off or add some more bookmarked topics to include those you care about.
  • You specified posts in specific forums only. Broaden the forums you want to see or check “All”.
  • You specified you don’t want to see more than X posts in a digest. Once you hit this limit, digests will automatically ignore any other qualifying posts.
  • You specified you only want to see posts that contain a minimum number of words. Make the number lower or set it to zero (0).
  • You specified to show new posts only. Only posts after your last visit date/time in phpBB will be shown. So if you visit the forum regularly during the day, this date/time will be captured by phpBB, even if you didn’t visit a particular forum and never read those posts.
  • You specified to show only popular topics only. If, for example, you have a setting to show only topics with five or more posts per day, only those posts will show. So if there are none that qualify, there may be no digest to send or it will be empty. Popularity is based on recent postings, so for a daily digest the topics must have five or more posts over the last 24 hours.
  • You asked to remove your posts.
  • You asked to remove posts from your foes.
  • You asked to be sent a digest even if there are no new posts. This is basically a “comfort” feature to let you know digests is still working but there’s nothing new you are interested in.

Forum administrators also have a host of privileges that can affect if you receive a digest and what’s in it.

  • They can change the day of the week for sending out weekly digests. Typically it’s Sunday, but they could make it Wednesday.
  • They can also set a cap for the number of posts in a digest. If set this will affect not just your subscription, but everyone’s. Any posts above the cap will automatically be ignored.
  • They can exclude forums from ever appearing in a digest. Conversely, they can also require one or more forums to appear in a digest. (This doesn’t apply if you selected bookmarked posts only.)
  • They can balance the load so that digests are sent more evenly during the day. This could change when you receive a digest, possibly temporarily causing some posts not to appear in digest (or even to appear in a subsequent digest.) They have the option to send an email in these cases, but they don’t have to.
  • They can change any of your digest settings in the Edit Subscribers function. This could cause you to not see posts that you used to see.
  • They can mass unsubscribe users. If you find your digest type is none and you didn’t change this, this may be what happened.
  • They can disable the forum. While it is disabled no digests will go out. Things should catch up once the forum is reenabled.

Some things may affect the timely delivery of digests. Ideally a forum administrator will create a “system cron” so digests go out hourly. If they don’t, then digests may be delivered late as a “phpBB cron” depends on board traffic to kick off digest processing. In this case you just have to wait. However, if you go to the forum you might kick it off and then receive your digest.

Most of these features were features that were requested by users over the years. The popularity of digests then is due in part to meeting the demand of the market.

Updated November 26, 2018 to discuss the new popular topics feature.

Why does my forum’s performance suck?

With growth or just with the passage of time, phpBB forums can slow down. Pages may take a while to load if they load at all. You might get HTTP 500, 403 or other errors.

Today, site visitors expect fast response and if they don’t get it they may just decide it’s not worth coming to your forum. However, troubleshooting the root cause of these problems can be hard.

Often just opening a support ticket with your web host will bring you resolution. Or not. Web hosts run the gamut from poor to excellent, and generally the less you pay the poorer the service is. Web hosts though often know what the problems are, they just don’t want to tell you, as it makes them look bad. They may want you to pay for a higher quality of hosting to “solve” a problem that is really their fault, because they don’t maintain their infrastructure properly.

Here are some reasons that may be at the heart of your forum’s performance problems:

  1. Increased traffic. The traffic may not be traffic to your forum, but to other websites hosted on the same machine (server) as your website, if you are using shared hosting. This used to be a huge problem. Hosts would overload servers with domains because it was cheaper than going out and buying new machines. This works for a while until enough customers complain. The smart customers rehost somewhere else but obviously their hosts are hoping the rest won’t bail because it’s too much hassle. If traffic is getting consistently slower over a relatively short period of time or you are seeing a consistent increase in 500, 403 or similar error pages while using shared hosting, this is probably the reason. If your site is getting massively more traffic than you did before you might need to upgrade your hosting to a virtual private server or possibly even a dedicated server. Web hosts usually provide reports on usage that will help you get an idea if this might be the cause.
  2. Old hardwareThe law of entropy states that over time systems must degrade, i.e. become more complex or inefficient. It happens to your host’s servers too. Your website may be on a server that is ten years old, or has insufficient memory or CPU power for today’s needier software. So work may go into a queue until the CPU can catch its breath. While you are waiting of course nothing is happening. One clue may be to check the software installed on a server. For example, if you look in phpMyAdmin and you find that your MySQL database is running version 5.1 (released in 2008) then maybe your server is 2008 vintage, or older. Today MySQL 5.5 is generally the minimum version installed, with 5.7 being the newest and shiniest.
  3. Underpowered servers. Your server may not really have sufficient resources (CPUs, disk drives, etc.) to handle the traffic that is on it.
  4. Resource limitations. With shared hosting in particular there are implicit and generally poorly understood limits on the number of “resources” you can use: files, memory, CPU time or units of database work. You will usually see errors like this when they happen. They mean generally two things: either you are outgrowing your hosting or your server is overcommitted with other domains it is supporting. If your traffic is pretty steady, it’s more likely the latter problem. This is a sign of a poor host. You might want to rehost.
  5. Dated software. I mentioned that a host still running MySQL 5.1 is a yellow flag of caution. It’s not that MySQL 5.1 was a bad version, but many new features and performance improvements have been added since 2008. Your forum’s software has probably gotten more complex so it could use those improvements. Similarly, hosts running PHP 4 (fortunately, there are very few of these left) or even PHP 5.2 need to update their technology stack. You can see your version of PHP, web server software, operating system and the database in the Administration Control Panel, PHP information. Even the web server software you use can be dated. Still on Apache 1.2? You should probably be hosting somewhere else.
  6. You’ve been hacked! This happens rather infrequently, but it does occur from time to time. Malware or other bogus code may have infected your phpBB programs, templates and stylesheets, chewing up CPU and worse possibly infecting your readers’ computers with malware. Needless to say any malware must be removed. It may be discovered by comparing your software with a reference version of phpBB that you are using.
  7. Your database needs tuning. With so many reads and writes, your database may have grown and scattered itself all over the server’s disk platters, making it time consuming for the database management system to read and write data. You can try repairing and optimizing your forum’s tables in phpMyAdmin or a similar tool and see if performance improves.
  8. Your database is not properly configured. You may be missing primary keys, or indexes for tables. In some cases certain standard columns (added with upgrades) end up missing. Primary keys and indexes are used to retrieve data quickly. If they are missing the database won’t complain, but it will read all rows in a table to get what it needs, which is highly inefficient and can make a forum crawl. These sorts of problems are often the result of database updates that never finished properly, although they may have said they did. This can happen frequently upgrading from phpBB 3.0 to 3.1.
  9. Your phpBB version is woefully out of date. I am still surprised by requests that I get to upgrade phpBB 2 forums. phpBB 3.0 was released in late 2007! The problem is often not phpBB 2 itself, but your PHP version, which has evolved, making the handshake between PHP and phpBB awkward and inefficient, if it doesn’t trigger PHP warnings or errors. Upgrades are never fun or painless, but something you need to keep up on.
  10. You need to use a different database. Typically phpBB is used with the MySQL database because it’s free. phpBB at least in theory supports a host of other databases including SQLite and SQLServer. Most of the testing is done against MySQL and really phpBB is optimized for use with MySQL or its clone MariaDB. You will probably see better performance if you stick with these databases.
  11. You are using Microsoft software on the server. IIS is Microsoft’s web server. If you are running IIS you are also on a server using a Microsoft server operating system. IIS is not a bad web server but it works suboptimally with phpBB. phpBB should work best in a standard environment. So if you have the option to move to a Linux/Apache web server you may find marked performance improvement.
  12. Your browser is old. phpBB expects a reasonably modern browser. If others aren’t having issues but you are, maybe it’s time to upgrade your browser from IE7 or Firefox 2. Stick with recent versions of browsers like Firefox, Chrome, Safari and Opera.

There are other reasons your forum could be slow, but most likely the cause is one or more of the above. If you don’t feel you have the expertise to troubleshoot these issues yourself, or simply would rather have it done professionally, send me a service inquiry.

April 2016 work summary

April was a slow month, at least compared to March which was pretty frantic. Here’s some of the work I did in April for clients:

  • Upgraded a forum from phpBB 3.0.11 to 3.1.8. I installed Tapatalk and Cleantalk extensions. I installed American English language pack. Added the board’s logo. Expect to do a rehosting job at a later time.
  • Upgraded vanilla forum from phpBB 3.1.6 to 3.1.8.
  • Upgraded a forum with non-referenced style from phpBB 3.0.12 to 3.0.14 with many modifications. I removed some unneeded mods by manually uninstalling them and removing the style changes for the mods too. I then uploaded a virgin copy of 3.0.14 overwriting everything for consistency. I reapplied all file changes to mods that he wanted to keep, upgrading them if there was an upgrade and re-uploading the latest programs for these mods. There were styling issues with the non-standard style. I then worked through various issues. Spambot countermeasures could not be selected. Repairing tables used for it made it work. phpBB Gallery comment photos bled over to the next comment. Increased the size of the container to 350 px to accommodate it. Memberlist profile would not show, turned out to be missing ENDIF in template, which I fixed. Also installed latest en_us language pack and made sure mod language changes were made to en_us.
  • A user tried to upgrade from 3.0.8 to 3.1.6 and there were errors. He asked me to investigate and fix them. Three tables needed columns added. I also had to add six rows to the config table. I manually updated the reported phpBB version in the database. Once at 3.1.6, I upgraded from 3.1.6 to 3.1.8. All looks good.
  • Short Skype conference with a client. I upgraded forum from 3.1.8 to 3.1.9. I gave client two reports: one of users with profile fields and a list of forum names with their parents so he could determine which users can be deactivated.
  • I tutored a client in how to send files via FTP to their domain.
  • I upgraded a forum from 3.1.8 to 3.1.9. No changes to elegance style with upgrade.
  • I upgraded a forum from 3.1.5 to 3.1.9. The client reported that some people weren’t getting activation emails for new accounts. I could find no evidence of a problem. I asked user to look into spam folder. There are regular activations in the board’s user log.