Blog

Solving phpBB forum permission issues

I have noted before that phpBB’s permission system is awesome. In one way though it’s a bit defective: it’s hard to troubleshoot issues with permissions, particularly forum permissions.

In this post, I’ll delve into solving forum permission issues. The general problem is that a user typically belongs to more than one group and different groups can have different forum permissions. If you are unfamiliar with the basic phpBB groups, you might want to read this post first. You can also create groups of your own and set forum permissions to those groups.

To solve these issues, you generally you need to start with a test case. This part at least is pretty easy because you usually have a user whose permissions are not working correctly, so you just need their username. You also need one or more forums where their permissions are off. You also need to know what permission isn’t working right, such as permissions to create new topics or reply to topics.

If you dig into phpBB’s documentation, it becomes a little clearer. On permissions, the documentation says:

  • YES will allow a permission setting unless it is overwritten by a NEVER.
  • NO will be disallow a permission setting unless it is overwritten by a YES.
  • NEVER will completely disallow a permission setting for a user. It cannot be overwritten by a YES.

So essentially when the NEVER permission is set, it becomes a blocker overriding any other permissions.

Seeing all forum permissions for a user, forum and permission type

How do you see these forum permissions? You need a tool. The good news is that phpBB has just such a tool. The bad news is that they bury it. In fact, it could not be harder to find. In this example, I will use my development forum to see forum permissions for myself.

  1. Go into the Administration Control Panel
  2. Select the Permissions tab
  3. On the left sidebar, go way down to the bottom. You want View forum-based permissions. Click the link.
  4. Pick the forum or forums you want. In this example, I chose the “Your first forum” forum created by default when you create your board. I then pressed SUBMIT.
  5. Now I pick the user. There are various ways to do this with the interface for both users and groups. In this case I choose myself, with a username of “Mark D Hamill”. I entered the username in the “Find a member” field and pressed the View permissions button below the field. This brings up the Viewing permissions page. The colors you see on the permission tabs may vary from mine where the green boxes in each of the tabs basically say “This user or group has YES for ALL permissions under this tab”. Red means “This user or group has NEVER set for all permissions under this tab”. Blue means “This user or group has a mixture of permissions for the permissions in this tab.”
View permissions page
View permissions page
  1. I now want to check out a particular permission. You may have to hunt for the permission you want to check for the user and forum as it may be on a different tab. There is a tiny little icon to the left of each forum permission. That’s what you need to click on. In this case I want to see how permissions are determined for the Can start new topics permission.
Selecting a permission to check
Selecting a permission to check
  1. This generally brings up a popup window. If you don’t see a popup window, you may have to tell your browser to allow popups for the domain. Finally this brings up a useful screen:
Viewing a particular permission
Viewing a particular permission

Now you can see what’s going on for this particular user’s forum permission. Since I am both an administrator and a global moderator, I belong to both those groups, each of which has a group forum permissions too. By default, NO access is allowed to the forum for a user, so it is the first permission. But it is set to YES for Administrators, so the logic continues and the next group is tested. It is set to YES for Global moderators too, so the net permission is still YES. It is set to YES for registered users in this forum too, so it’s still YES. Finally, it looks for any user-specific forum permissions. None were granted, so this permission is NO, but since it is NO and not NEVER the overall YES permission still applied.

Fixing the underlying permission issue

With this tool, you should be able to determine where the root of the permissions issue lie from the variance from an expected permission from the actual permission. It’s usually a group permission that needs changing. The most likely solutions are:

  • A NO permission should be NEVER
  • A NEVER permission is blocking everything so it should be changed to NO
  • A NO or NEVER permission is incorrect and should be YES

So adjust the group or user permissions for the forum privilege accordingly. You can use this tool to check to see if the result is correct, or use the feature in the Administration Control Panel to test out a user’s forum permissions.

Usually in these cases, you cannot use the built-in forum roles. Rather you have to click on the Advanced link for each forum and group and change permissions that way instead.

In some cases, you can adjust the permissions for a forum role and have them trickle down accordingly, avoiding the need to use the advanced link when setting forum permissions. I’ll leave you to investigate this option if you want instead. It can get a little hairy to change these because it affects all forums where these permission roles are used.

Setting user-specific forum permissions is always a bad idea. Remove them if you can, and place these forum permissions in groups you create instead. Add people to these groups as necessary to get the desired behavior.

The newly registered users group permission quirk

There are some things that are definitely peculiar about phpBB’s permission system. Newly registered users are also in the registered users group. To start, this makes no sense. In the case of this user, “tester66”, because he is in the newly registered users group, the forum’s permissions for newly registered users does not allow them to start new topics. But because they actually are in the registered users group too, they can start new topics, the exact opposite of what you would expect!

Newly registered users permission quirk
Newly registered users permission quirk

How do you solve this problem? You have to set the permission to NEVER for the newly registered users group by using the group forum permissions function. In it you select the Advanced link to fine tune the permission. After changing the permission, you can see the result:

Fixing the newly registered users group quirk
Fixing the newly registered users group quirk

 

January 2019 work summary

I was on vacation January 5 – 18 so you would think work would slack off this month. But while I was gone a queue of work developed. I was kept reasonably busy when I wasn’t touring Ecuador and the Galapagos Islands. In short, despite taking two weeks off, I made my revenue goal for the month thanks to customers who waited patiently for my return.

  • An old phpBB 2 forum on an old free host was somehow upgraded to a newer version of PHP. That caused the old forum to mostly stop working. Many screens were garbled or content was hidden or inaccessible. I was able to infer the phpBB 2 URL for downloading the database by doing a test on my laptop. This allowed me to download the database, but the avatars (since they were images and no FTP was available) remained inaccessible. I was then able to convert it to a phpBB 3.2.4 database which I placed on my domain in a folder for the client to inspect. Currently the client has a converted database that they can move to any host. They have not asked me to take this next step, at least not yet.
  • Updated a forum from phpBB 3.2.3 to 3.2.5. When I tried to make a database backup, I downloaded it and discovered that it was not complete. This was due to annoying shared hosting timeouts on Bluehost, which really metes out resources sparingly to clients, one of the reasons I urge clients to avoid companies owned by the Endurance Group like Bluehost. I eventually made a set of backups by making slices of certain tables using phpMyAdmin. In short, this relatively simple upgrade was something of a pain, mostly because I won’t do an update unless I have a complete recovery path. I had to change the logo size in common.css to put the old logo back. The client’s Advertisement Management extension needs an update, which I later updated.
  • Troubleshooting. A client complained about spam they were getting. This is probably because Contact form was enabled but the enabled Cleantalk extension was not checking it. I enabled Cleantalk to check it. Next was a complaint about users not being able to pass reCaptcha. I created a new set of reCaptcha keys, but this one used the checkbox Captcha that phpBB 3.2 supports, and deleted the old one. Their [youtube] and [vimeo] BBCodes were not embedding properly. I installed the Media Embed extension and disabled the [youtube] BBCode by renaming it so I could use it with Media Embed extension. Vimeo has similar issue but I left that untouched. 248 Youtube and 105 Vimeo embeds were affected. I noted that anyone could edit their own posts to fix the embed issue and provided instructions.
  • What looked like a newly upgraded phpBB 3.2.5 forum needed styling work to make it match a larger website (they were using prosilver). After going through the available styles with the client, he agreed to and I installed the ne_blackgreen style which I then proceeded to tailor to make it match the larger website. I concentrated on header and footer changes to the pages. For the header, the client wanted three image areas across the top, evenly spaced, left side pushed left, center centered, right pushed right, of equal size. I made it responsive … was in previously in a HTML table that was not responsive and did not look good on mobile devices. I installed the NavBar Search extension to put the search bar on the navigation area so I had the full header available. For the page footer, I was asked to add some standard text. I removed many HTML validation errors from using old HTML. I converted the old HTML tables to responsive <div> tags. I changed the PHP version from 5.6 to 7.1. I tried to change the spambot countermeasure to use reCaptcha but it gave a HTTP 500 error, so I reverted to Q&A spambot countermeasure. Forum pages are now responsive. The larger website is still not responsive. This could be take up as a future project. I changed user registration settings to allow people to respond to an email link to complete registration. I changed the base font size to 13px. Added 1cm padding to the top and 2cm padding to bottom. Later, I did another hour of style tweaks, principally to make it look good on mobile devices. There were annoying issues with CloudFlare and no way to refresh its cache, so I had to wait to see changes. Cloudflare was eventually removed by the client.
  • I updated a forum from phpBB 3.2.3 to 3.2.5. I had to reapply one style change to common.css, otherwise it was an easy and normal update with no issues. Afterward, there was an issue with a phpBB Gallery error. I removed the cache file that reported the error and the error went away. I updated domain from PHP 7.0 to 7.2 so user avoids extra fee.
  • I upgraded a forum from phpBB 3.0.13-PL1 to 3.2.5. I started with the default prosilver style, but soon moved to Orange BBE style. I made some customizations. I moved the search interface to navigation bar. I had to resize the logo to 50% of its previous size to make the logo responsive to mobile devices. At screen width of under 700 pixels, I swapped in a logo that was one third of the original logo size. I installed the advertisement management extension because the client was previously using the mod and had ads to be ported. Because MySQL tables existed from the mod where the old ads were placed, there were some tables and columns missing causing some errors when the extension ran. It took some analysis to add what was needed to get it to work as an extension without losing the old ads. Basically, I had to add some columns and indexes. I replicated email, FB and donation links/buttons on the navigation bar. I replicated the background image in the style. I installed the mChat extension. I setup reCaptcha V2 as spambot countermeasure. I disabled the contact page as it is a channel for spam otherwise. I changed PHP to 7.2 for all domains. This client had the same UK host as the previous clients, and the host charged extra if they weren’t using PHP 7.2.
  • This client has a busy site and wanted to do an upgrade from phpBB 3.0 in stages, starting with development environment upgrade that had a copy of the production database. I had to use SSH and FTP to do this work. There is no cPanel or Plesk environment available. In the development environment, I upgraded phpBB from version 3.0.12 to 3.2.5. The database upgrade program look about 3 hours to run, and I had to manually create the phpbb_config_text table first. Next, there was style work to match current style, about 3 hours of labor for that. I installed the Tapatalk, Thanks for the posts, Digests, ACP Add User and Google Analytics extensions. I disabled the contact page. I enabled the reCaptcha V2 spambot countermeasure. I added a home link to the navigation bar. While I installed my digests extension (they have the digests mod installed), I did not set up digests cron, but users’ subscriptions were successfully carried over. The Tapatalk extension may not work correctly until moved to production because it’s in use in the production forum. No ForumRunner extension was available so that functionality was lost. I installed the PM Search extension, a release candidate extension. The Thanks for the posts extension may need permissions set up. The client will be doing some testing of the development environment. When complete, I will upgrade the production forum. That work should show up in next month’s report.
  • Someone sent me a $50 donation for my digests extension, a thank you for finally getting it approved (a 3 year effort!) I had no way to thank them for the donation, and I tried to give it to the phpBB group but could not find a way to donate money to the phpBB group, so I kept it. Thanks, donor!

Integrating your phpBB topics and posts into WordPress … or any web page!

As some know, I am the developer of phpBB’s Smartfeed extension. This extension provides an ATOM, RSS1 or RSS2 feed of posts and topics on your website.

These feeds allow people to read posts on the forum remotely using a newsreader application, like one built into MS Outlook or using feed aggregator sites like feedly.com. The main advantage of feeds is that it allows you to read a forum without actually having to visit the site. If you regularly read lots of sites, using a newsreader is very efficient way to read content compared to actually visiting each site.

Smartfeed is not the only extension that does this. In fact, if you don’t need to support the RSS format and only want to show posts in public forums, an ATOM Feed has been built into phpBB since version 3.0.6. You might want to read the knowledge base article to learn the syntax to use. You can control your feeds in the Administration Control Panel: ACP > General tab > Board configuration > Feed settings.

Sometimes you want to highlight recent topics and posts on your larger website, or on another domain. For example, you may have a phpBB forum in a folder on a WordPress site. You might want to use a WordPress widget to highlight current topics and posts on a sidebar of your WordPress site. The links in the sidebar will take people directly to the post or topic of interest.

I will demonstrate how to do this using WordPress. However, conceptually you don’t need WordPress to do this. You just need something that can read an ATOM or RSS feed of your forum, and parse its XML into HTML for display, or write your own program to do this. For example, if you are familiar with jQuery, there are a number of jQuery feed plugins that would work. The PHP SimpleXML library is one way you can do it in PHP with a short PHP program.

Here’s one way to do it in WordPress:

  1. Spend some time figuring out what you want to highlight in WordPress: recent posts or recent topics. I will show a list of recent posts. In this example, I first installed my Smartfeed extension. This is because I got a SimplePie parser error when I used phpBB’s ATOM feed. This is due to a bug when parsing ISO dates in ATOM feeds using the SimplePie library. SimplePie is bundled with WordPress. I reported the bug. The resulting URLs for the feed can be seen if Smartfeed is installed. It is in the HTML source for the forum. I will use the second link because the ?y=2 parameter creates a RSS feed instead of an ATOM feed to avoid the SimplePie bug.
<link rel="alternate" type="application/atom+xml" title="ATOM" href="/phpbb/app.php/smartfeed/feed" />
<link rel="alternate" type="application/rss+xml" title="RSS" href="/phpbb/app.php/smartfeed/feed?y=2" />
  1. Note: if using Smartfeed, and you want a list of recent topics only, the resulting URL will look something like this. You should be logged out when using the Smartfeed interface. Note that you can refine the URL in the Smartfeed user interface. lp=1 limits the feed to the last post in the topic only, t=2 suppresses the time limit for retrieving posts, s=1 gives a standard sort from most recent to least recent, i=0 means to not require a minimum number of words in the post, y=2 forces a RSS2 feed, d=3 sets the feed style to use HTML, w=0 means not to limit the maximum number of words in the post, and tt=1 means to show topic titles only. There are lots of variations so use the Smartfeed interface to get the output just the way you like it.  
http://127.0.0.1/phpbb/app.php/smartfeed/feed?lp=1&t=2&s=1&i=0&y=2&d=3&w=0&tt=1
  1. Presumably you have installed WordPress already. If you haven’t, it can be downloaded from wordpress.org. Or you can usually install it from Plesk or cPanel.
  2. Login to WordPress as an administrator and go to the WordPress dashboard.
  3. WordPress comes with a RSS Widget preinstalled that also can handle ATOM feeds. You can find it off the dashboard: Appearance > Widgets
  4. Click on the RSS Widget then press the Add Widget button which appears, which by default appears on the sidebar.
  5. In the sidebar, click on the RSS Widget that was added. Enter the URL for the feed and give it an optional description. Note that it needs to be the full URL, not the partial one shown above. Then press Save.

    Configuring RSS Widget
    Configuring RSS Widget
  1. I dragged the widget to the top of the sidebar so it would appear first on the sidebar. Of course, you can place it anywhere on the sidebar that you like.
  2. Go to your WordPress site and find it on the sidebar.

    List of recent forum posts in the WordPress sidebar
    List of recent forum posts in the WordPress sidebar

Note that this works for any domain, providing the feed is publicly accessible. So you can promote this approach to have similar sites show your list of recent topics or posts. Note also that my Smartfeed extension has a number of options to make the post subject or topic title prettier if the default looks too wordy.

Digests extension approved

It took three years, but the phpBB extension review team has approved my digests extension. Given its complexity, my less than stellar skills at programming in the new extensions architecture, and the long lag time for getting a review, it’s not too surprising that it took so long.

The extension can be found here.

Further support questions should be addressed here.

 

December 2018 work summary

Here’s a summary of the work done for clients in December 2018. It was definitely busier in December compared to November, with peaks and valleys.

For 2018, revenues were up considerably, more than I estimated: 27% more in fact. Looking over 2018, I did a total of 126 individual jobs, many of them multiple jobs for the same clients. Since I started the business in 2006, I have done 991 jobs in total for 356 distinct clients. As always, unless a client chooses to reveal themselves in a comment, I keep information about the client confidential in these work summaries.

  • Tested upgrade from phpBB 3.2.1 to 3.2.4 on development forum. Style is inherited from prosilver and had many changes. Compared prosilver changes since 3.2.1 to see what additional template changes were needed. Applied them. Customer tested, made a few changes. Proceeded to a production upgrade. Issues trying to backup files and database. Had to sort through credential and SSH issues. Updated two extensions: Pages and Stop Forum Spam. Chose not to upgrade PHP to version 7. Provided suggestions on performance issues. Implemented 3.2.4 bug fix. Installed Posting template extension. Fixed a style complaint with Your Posts, the page number buttons were aligned toward the top. Commented out the CSS in the local style. First I performed a test upgrade from phpBB 3.2.1 to 3.2.4 on development forum. The user’s style was inherited from prosilver and had many changes. I compared prosilver style changes since phpBB 3.2.1 to see what additional template changes were needed, then applied them all. Customer tested, made a few changes. I then proceeded to a production upgrade. There were issues trying to backup files and database. I had to sort through credential and SSH issues. I updated two extensions: Pages and Stop Forum Spam. Client chose not to upgrade PHP to version 7 at this time. I provided suggestions on performance issues. I implemented some phpBB 3.2.4 bug fixes. I then installed Posting template extension. I fixed a style complaint with Your Posts: the page number buttons were aligned toward the top. Commented out the CSS in the local style.
  • Upgrade from phpBB 3.0.10 to 3.2.4. Applied patches to 3.2.4. Installed prosilver SE style, widened it to 80% max width. Installed mChat extension. Had to create a table to complete the upgrade. Disabled contact form. Configured reCaptcha. Placed logo. There was a big issue. Could not get into ACP, complained about being unable to convert UTF-8 to UTF-16BE. Found the code and made it lie so rather than fail it returned true. Then I was able to get into the ACP and finish the work. To install mChat, had to remove a module and some migrations. I upgraded a forum from phpBB 3.0.10 to 3.2.4. I applied patches to phpBB 3.2.4. I installed prosilver SE style, widened it to 80% max width. I installed the mChat extension. I had to create a table to complete the upgrade. I disabled contact form, then configured reCaptcha and placed the logo. There was a big issue. I could not get into Administration Control Panel. A PHP library in phpBB’s vendor folder complained about being unable to convert UTF-8 to UTF-16BE. I looked on phpbb.com and there was nothing quite like this issue reported. This is probably some obscure table or column encoding or collation issue in the converted database. I found the code that triggered the error and made it so rather than fail it returned true. Then I was able to get into the ACP and finish the work. To install mChat, had to remove a module and some migrations.
  • This was probably the big, hairy job of the month. I upgraded 1M post, 900K user forum from phpBB 3.0.10 to 3.2.4. Upgrading forum took about 12 hours. Could not upgrade with CLI, tried three times. Continued manually with many timeouts. Meanwhile, moved the database to my machine and converted it there as a backup process. Very tedious. Had to create two tables: config_text and oauth_states. Eventually finished and was eventually able to use CLI. Incorrectly applied patch resulted in topics not being seen correctly. Installed Latte style. Applied logo. A new style is in development. This is temporary. Disabled contact form. Installed American English language pack. Set up reCaptcha V2 spambot countermeasure. Set up Home link on navigation bar. 12/9 – 12/11: Back and forth on styling issues. Solved by user supplying a new style that I enabled. Some issues with the upgrade were noted. Provided feedback. One issues with header errors in ACP > System > General tasks looks like a phpBB bug. Reverting PHP from 7.2 to 7.1 would solve the issue.
  • Client tried to update phpBB 3.2.2 himself to the latest and a HTTP 500 error occurred. Wasn’t sure the update was complete. Spent a lot of time trying to figure it out. Eventually determined extensions were not disabled first and that an old version of the Board rules extension triggered errors in the error log. Uploaded a new version, cleared the cache and the 500 error went away. Earlier ran the update program and after completing it successfully and going to any URL the error occurred. Lots of extensions were out of date so I also updated: pages, stop forum spam, user merge and mchat.
  • Continuation of work after several months. Upgraded forum from 3.2.2 to 3.2.4. Moved over latest code from mailing list extension on asoprsforum.org. Created a Subscribers user group (group_id = 8). Edited mailing list code to use this group_id and to remove condition for a column that does not exist on this host. Changed the logo to use similar font size and font face for site name and site description. Tested new version of mailing list extension. Works fine but may have issues with larger group sizes. Changed attachment.html on prosilver to replicate changes there. Increase since of content to 1200 pixels to make embedded videos look better. Installed Digests 3.2.10 and tested manually and automated. Set client up for a daily digest as requested. I finished a project I had been working on with a professional association that I began in August for a new forum on a new domain. Basically I replicated the work I did for them on another domain to make it work the same. This included some unique functionality. For example, phpBB cannot play web ready videos natively, so I hacked attachments_body.html to natively play .mp4, .m4v and .ogg videos natively. phpBB requires these videos play using a Quicktime plugin. Tested on a variety of devices including smartphones to make sure it worked on all devices. Some devices seem to require a Chrome browser. Since I started it in August with a prototype, phpBB has evolved. So I upgraded the forum from phpBB 3.2.2 to 3.2.4. I moved over latest code with my changes from the mailing list extension on the other domain and tailored it to work on the new domain. I created a Subscribers user group used to send notification emails of new topics and posts (with full texts) and placed all registered users in that group. I changed the logo to use similar font size and font face for site name and site description, and eventually installed a total of five styles that the user could change on the navigation bar. I tested new version of mailing list extension. I installed Digests 3.2.10 and tested manually and via a cron. I set it up for a daily digest as requested, but disabled the user control panel interface as requested. The client is the only one getting the digests. Also with the ACP Add User extension enabled, I added a few dozen members to set up their accounts. The forum is hidden from the public.
  • Troubleshooting. Client reported 403 (forbidden) error messages. Hosts did something to fix it, said it was an issue with app.php. Changed four folders to use 777 permissions, were 755. Looked at error log. There was a deny all line at the top of the .htaccess file. Removed that and the forum came up. No obvious problems with the board. Later, updated phpBB 3.2.2 to 3.2.4. No issues with update.
  • Upgraded phpBB from 3.2.2 to 3.2.4. Applied patches to 3.2.4. Upgraded Cleantalk extension to the latest version. Updated latest American English language pack. Made sure all changes to prosilver were applied: ads are supposed to show in the header. I upgraded phpBB from version 3.2.2 to 3.2.4. I applied the needed patches to version 3.2.4. I upgraded the Cleantalk extension to the latest version. I updated the forum to the latest American English language pack. I made sure that all changes to prosilver were applied: ads are supposed to show in the header.
  • Upgraded phpBB from version 3.1.11 to 3.2.4 with patches applied. Upgraded Cleantalk extension. Tried to install upgraded Custom Code extension but it requires PHP 7 not yet enabled. When enabled, the extension should be enabled. Updated Google Analytics extension by mistake. Disabled Auto DB backup extension and email list extension (version 3.1 base). Installed American English language pack. Reinstalled logo and adjusted display size in CSS.
  • Client complained that old embedded images hosted on other websites were showing blank spaces instead. When this happens, it’s because the image no longer exists on the external web server. However, there was another thornier issue. The client did not want users to use the [img] BBCode, but did want admins to be able to use it in posts. The general problem was that user forum permissions were being granted to admins, instead of adding users to appropriate administrator’s group and giving this permission to anyone in the admin group. Those user forum permissions had the Never permission set for using the [img] BBCode. I changed the forum permissions for registered users for this privilege from Never to No. When set to never, since admins were in the registered users group, this would also keep them from using the [img] BBCode. However these user forum permissions for these admins were set to Never and overrode this change. It was hard to tell how many users were affected since admins were not actually in the administrators users group. Having their user status set to founder allowed them to get into the ACP but the privilege to place others in the administrator’s users group returned an error. Using the database, I determined that likely no more than four people had this issue. So I changed their permissions for this privilege set to Yes from Never for all forums, which was time consuming because there are many forums. I provided advice on how to allow posts to be edited for up to six hours (30 minutes was the default) on the post settings page.

November 2018 work summary

This was a slow month for my business. It wasn’t a problem. I made my goal for the year at the end of October and was also teaching a class. Good to have some downtime. It allowed me to get out a new version of my digests extension.

But I did do some scattershot work here and there in November, updates mostly, including this work:

  • Upgraded a forum from phpBB 3.2.1 to 3.2.3. Initially I could not login to the Administration Control Panel. Changed cache folder permissions to make everything writeable and eventually was successful. I had to copy all the files from a version 3.2.3 reference to get the updater to work due to TWIG library errors that occurred when I tried to update the database. I reapplied the forum’s style changes for the logo and a PayPal button. I updated the Advanced BBCode extension from version 3.1.3. to 3.2.1. Changed all tables to use the InnoDB storage engine instead of the old MyISAM storage engine. There was an issue with the topics table. I had to remove a MyISAM full index before I could change its storage engine.
  • Additional work for a client I mentioned last month. Last month I completed a long upgrade of their forum from phpBB 3.0 with many modifications. In this latest work I changed the mailing list extension to use a new “Subscribers” group I created instead of Registered Users for sending topic and post notifications. This made it easier to opt out people who did not want these emails by simply removing them from the group. I created the new group then populated it with a SQL query by adding everyone in the Registers Users group to it. I provided instructions on how to remove people from this group. I noticed an inconsistency in phpbb_user_group table, removed lots of bogus rows where the user_id in the table did not exist in the phpbb_users table. I manually removed a bunch of people who did not want these notifications from the new group. A few days later, I refined the query to also filter out those where a column I created called user_email_all_posts was set to 0. I used this approach when a similar functionality was done on phpBB 3.0. All the notification issues seem to be fixed now and no latency has been reported due to emailing when creating new posts. Phew!
  • Troubleshooting. A catchable fatal error occurred on a client’s forum. It took a while to get access to the web host control panel due to the client’s illness. Once I had it, I manually purged the cache. This generated a different error. I changed PHP from 5.6 to 7.0 though made it go away. I changed permissions on the cache folder too. Later I updated the forum from phpBB 3.2.1 to 3.2.4. When I tried to run the update it said phpBB was not yet installed! I searched through the phpBB source code to see what triggered the error, eventually discovering it was because a .lock file was present in the cache folder. I removed it and the update was successful.
  • Updated forum from phpBB 3.2.3 to 3.2.4. No issues. Later, I installed the mChat extension on the forum.
  • Updated forum from phpBB 3.2.1 to 3.2.4. Updated the AllanStyle-SUBSILVER style that was the primary style, putting back the old logo. I removed the old security certificate, and installed a free Let’s Encrypt certificate. I changed the forum’s .htaccess file to redirect HTTP traffic to HTTPS. I discovered one issue on the index: insecure content was being served. This was due to http:// being in the user_avatar column of the phpbb_users table to serve avatar images. Used MySQL REPLACE to make these avatars to be served securely. In some cases there may be a blank space now because the server did not support HTTPS. I changed PHP from 5.6 to 7.2. This work inspired this blog post.
  • Updated a forum from phpBB 3.2.2 to 3.2.4. Upgraded the Cleantalk extension to the latest version. I changed config.php to use mysqli. This allowed the forum to work with PHP 7.1 (it was on PHP 5.5).

Fixing insecure content issues in phpBB

Updated December 27, 2018 to correct some things based on new information.

So you’ve decided to use HTTPS for your forum to show your content securely. This is good and it’s not too hard a thing to do in most cases. Everything looks good but sometimes you notice on browsers like Chrome the little green lock icon up on the URL field disappears. What’s going on? If you investigate by clicking on the icon you can usually figure out what’s going on: there is some insecure content on the web page.

What is insecure content?

Insecure content is content embedded on a web page that is delivered insecurely, i.e. from a web server using http instead of https. Usually these come from external sources, and are typically externally hosted images that are served insecurely.

One way to investigate these is to view the HTML source of the web page. Use the Find feature to scan for URLs with http:// instead of https://. The issue occurs with embedded images like this:

<img src="http://www.externalwebsite.com/myavatar.jpg />

If all these URLs could be changed to something like:

<img src="https://www.externalwebsite.com/myavatar.jpg />

then all would be well, that is if the external website supports https.

How do you fix these problems? There are typically two places where these problems manifest:

  • In post text
  • In the user’s avatar

Here are some approaches you can use to solve to fix the problem:

Use the Image Redirect extension

As of this writing the Image Redirect extension is a Beta release, so it is not recommended that you install it on a production system. This extension also requires that you set up a proxy server on your web server, not a trivial tasks and something you may not be able to do on your class of hosting. Camo Proxy is one example of a proxy server you can install. What this extension does is scan the page for these external image URLs, fetches them using a proxy and changes the URL so that it is served from your proxy copy, which will be on your machine and served securely. In theory this extension should solve all issues like this. Note that it takes some time to create a proxy image if it is not cached and this adds some small overhead, which may slow page rendering.

Fix the embedded URLs in your database

This works by changing the URLs in your database. You scan for http:// and replace it with https://. Using this approach has some limitations:

  • The server serving the remote content may not have https installed. What generally happens is the image is not served and a white box appears instead. This could make lots of posts look off or unacceptable, particularly if these images are large.
  • While it corrects existing URLs, it doesn’t prevent someone from doing the same thing in the future.

If you can live with these limitations, you can fix it in the database. This approach assumes you have MySQL or MariaDB as your database and that the REPLACE function is available. It also assumes you have phpMyAdmin or a similar way to issue SQL (Structure Query Language) commands to the database. In phpMyAdmin, there is a SQL tab where you can type in and execute SQL. Just make sure you use a SQL tab for your database.

There are two tables that typically need fixing: phpbb_posts and phpbb_users. Steps:

  1. Disable the forum
  2. Backup the forum’s tables. Make sure it is a complete backup by downloading the extract, uncompressing if if necessary and looking at the end of the file. There should be SQL in there populating the phpbb_zebra table at the bottom of the file.
  3. Use phpMyAdmin or a similar tool to go into your database. If you are not sure which database you need to modify, look at your forum’s config.php file. The database name is in the file.
  4. You can examine the extent of the problem by first looking at each table. In these examples I assume your table prefix is phpbb_. The config.php file contains the actual table prefix, which may be different.
SELECT post_text FROM phpbb_posts WHERE post_text like '%IMG src="http://%';
SELECT user_avatar FROM phpbb_users WHERE user_avatar like '%http://%'
  1. To actually fix these, use the following SQL:
UPDATE phpbb_posts set post_text = replace(post_text, 'http://','https://') WHERE post_text like '%IMG src="http://%';
UPDATE phpbb_users set user_avatar = replace(user_avatar, 'http://', 'https://') WHERE user_avatar like '%http://%'
  1. Reenable the board
  2. You might need to purge the cache, but it should not be necessary.

After these steps, some users may notice that their avatar no longer serves and there is a big, ugly white space instead. They may try to change the URL in their Avatar settings back to http:// to restore it, in which case the problem may recur. This option can be disabled (see below). In general they should be encouraged to upload an avatar so it can be served from your web server, which will then serve it securely.

Preventing future insecure content

For avatars, the issue is due to allowing remote avatars. This can be changed: ACP > Board configuration > Avatar settings > Enable remote avatars > No

For posts, you can remove the permission to use the [img] BBCode. The easiest way to do this:

  1. ACP > Permissions > Group forum permissions > Registered users group > All forums
  2. For each forum, click on the Advanced permissions link, then the Content tab.
  3. Set Can use [img] BBCode tag permission to Never. Note: this will affect everyone, including special groups and administrators. If you want to have it affect only registered users, set it to No instead. Other groups however may retain the permission to post embedded images. You may want to use this pattern on other groups you have defined. 

Fixing blank spaces where embedded images should appear

Since blank space represent placeholders for external images that no longer exist, the URL may need to be corrected. You can try the MySQL Replace function above if you know the new pattern to use.

Alternatively, you can install the External Images as Links extension. This will substitute a clickable URL for the image. It’s likely the URL will lead to HTTP 404 error (not found), but it at least resolves the blank space image in the post.