So I’m starting to port my extensions to phpBB 3.3.
I’m surprised by how easy it has been, at least compared with porting my extensions from phpBB 3.1 to 3.2. This is because not much has changed between 3.2 and 3.3.
In moving from 3.1 to 3.2, there were all sorts of changes under the hood that made porting extensions challenging. The way posts were encoded in the phpbb_posts table was changed, which was a big deal. FontAwesome support, which was very limited, was introduced, meaning we could use these in our templates instead of embedding images. A new TWIG library meant that the old way we extension authors inserted dynamic data into pages changed pretty fundamentally. We were asked to use a new template syntax in our templates. Template library calls also changed quite a bit. In short, 3.1 to 3.2 required quite a bit of reengineering, both of the phpBB base code but also of its extensions.
For 3.3, there’s not much upgrading of the database needed. Only one table is altered: phpbb_users, to change the properties of the reset token. The content in the phpbb_extension_groups table is changed. The Flash media group finally goes away (Flash is obsolete) and any non-Flash files in the group are now assigned to the downloadable files group.
As noted, you can use more Emoji codes in 3.3 and can put Emoji in a topic title. But this doesn’t alter the database. I have to test for these in my extensions, but so far it’s not a problem because the existing function I used to display these fields handles it for me. So nothing to do there.
So phpBB 3.3 feels more like an update to phpBB 3.2 than it does a new minor version. On the whole, the new functionality is quite minor. The main thing I have to do is test with PHP 7.4, since phpBB 3.3 now supports PHP 7.1.3 to 7.4 only. Getting my extensions to a stable 3.2 place before upgrading PHP is turning out to be much more work than retrofitting my extensions.
You can read the release announcement here. Obviously, its biggest feature is that it works on phpBB 3.2 and 3.3. This is the first of my extensions to be updated for phpBB 3.3.
It can be downloaded from my Smartfeed page or from the GitHub branch. If downloaded from GitHub, make sure to place it in the /ext/phpbbservices/smartfeed folder. Disable the extension first, removed the old files, then upload the new files and reenable the extension.
At the end of December 2019, MaxMind, the publishers of the free GeoLite Country Code database used by this extension, required that users authenticate to download the database. So this version addresses this new requirement, but adds no new functionality.
You will have to get a license key to download the extension. The link you can use to get this key is in the extension’s settings page when you install this new version. You do not need to pay to get a license key. In fact, this version will only download the free database.
It will not work on phpBB 3.3, but I am told if you hack the ext.php file it will install and work correctly.
With this and my digests extension now fixed with their backlog of issues, I should be able to install a local PHP 7.4 development environment so I can test all my extensions for phpBB 3.3 and offer versions compatible with phpBB 3.3.
It can be downloaded from the extension’s page or from the GitHub branch. If downloaded from GitHub, make sure to place it an /ext/phpbbservices/filterbycountry directory.
Two days ago, the phpBB group released its latest minor version of phpBB: 3.3, also known as Proteus. You can learn more about it on its launch page. To give you some perspective, phpBB 3.2, the last minor version, was released on December 9, 2016. So it’s been three years since the last minor release of phpBB.
I looked at a development version a couple of weeks back. So I was kind of taken by surprise by 3.3’s sudden release. Minor versions tend to introduce some new functionality, and Proteus does. It’s just that for most administrators and users, it won’t seem like that big a deal and things will look and behave pretty much the way they always have. As with phpBB 3.2’s introduction, most of its changes are covert, rather than overt. Unless you know what you are looking for, you won’t notice much.
One hard-to-miss feature, at least if you use the default proSilver style, is the phpBB logo is different. It’s now a Scalable Vector Graphic (SVG), which makes it look crisp and shiny in all resolutions, including retinal displays. It looks a tad bigger, but also more white and almost glossy. Also, the logo includes the words “forum software”, which is new. Previously, the logo was a transparent GIF and it said “Creating communities”.
Updating is getting easier
Updating phpBB is getting easier too. It won’t compare to updating WordPress, which takes place entirely behind the scenes and can be done with a single click. The exact mechanics of how it will work is unknown until 3.3.1 is released. But the launch page says:
“With our brand new installer updating will be easier than ever in phpBB 3.3! Upload a single folder to your board and all your files will automatically be replaced.”
This will be welcome because updating has always been a hassle. Over time, it may affect my income a bit since a lot of it comes from updates. I expect a lot of my customers will still want me to do this as a service.
You will still need to upload one file, an archive. I also expect there will be a number of other manual steps, because there will always be issues of overwriting custom changes to styles and extensions that may have issues. You will probably have to back up your styles and extensions folder manually before updating. Time will tell.
New PHP requirements
Proteus requires PHP 7.1.3 or higher, and cannot use a version of PHP greater than 7.4. So many administrators will have to upgrade PHP first, which may be an issue for those using PHP 5.4 and 5.6. They will finally have to take the plunge.
Most likely a lot of these boards will have an issue: they will need to edit their config.php file to tell phpBB to use mysqli drivers instead of mysql drivers. So far, fixing this issue has not been intuitive.
Overall, taking the plunge to PHP 7 is good: twice the performance compared with PHP 5 and phpBB can use many new features in PHP 7 too. I have noticed some extensions have issues with PHP 7, however, for example the AWS S3 extension.
Upgrading from phpBB 3.2
The upgrade process from 3.2 is pretty much unchanged from 3.1 to 3.2, and will be more manual in nature than the newer upgrade process. You can see the steps required here.
Improved Emoji support
From a user’s perspective, the exciting feature is likely to be increased Emoji support. Previously, only certain Emoji characters could be used, and only in post text. Now you can use virtually any Emoji character, and you can use them in topic titles too. However, you cannot use Emoji in the subject line of topic replies.
The Emoji in topic titles permission is enabled by default. If you want to disallow it, the easiest way is to change a user role, like Standard Features. ACP > Permissions > Permission roles > User roles > [Role name]. Click on the green wheel for the role. See illustration:
Over time, phpBB forums will look a lot more colorful and visual.
No support for IE before IE11
Some other features they are highlighting:
Clever quotes. Quotes can show a link to the post and post author. It can also show the date and time of the quoted post.
Improved reCAPTCHA. Previously only reCAPTCHA V2 Checkbox was allowed for a reCAPTCHA solution. Now you can use the Invisible reCAPTCHA V2. One consequence of this is that the V2 Checkbox is no longer supported, so as part of upgrading phpBB to 3.3 you should have to get a new set of reCAPTCHA keys from Google’s reCAPTCHA site that support this method, and plug them into the Spambot Countermeasures area in the ACP.
Notifications are supposed to be very fast now. The whole notifications process has been reengineered. It’s unclear if this means email notifications are sped up. I’m pretty sure they will go into phpBB’s mail queue like they do now, so your Email settings should apply.
FontAwesome improvements. In phpBB 3.2, phpBB supported a rather limited set of scalable FontAwesome characters. The number supported are now much larger, and they will all look fine on retinal displays.
Symfony 3.4. This is behind the scenes stuff, but phpBB 3.3 uses a newer version of the Symfony PHP libraries, including its heavily used template engine.
ACP Statistics screen is now responsive. As noted in my first look, the statistics panel in the ACP now splits statistics into two groups, which has the benefit of making the screen responsive. You can see the new look below:
Should you upgrade?
You probably don’t want to upgrade right away. This is because some of your extensions may not work and if you made changes to your style, those won’t carry over so they will need to be replicated.
However, the same day the phpBB Group also released phpBB 3.2.9, which brings over some of these features including Emoji support. You might want to update to that version for a few months until extensions and your style becomes compatible with phpBB 3.3.
New year, but this also gives me an opportunity to review work I did for client in January and look at how I did overall in 2019.
Revenue-wise, it looks like 2019 was about as profitable as 2018. With a few quiet months, it didn’t look like it would be that way. How well I did depends on how you count the income. If you count it based on the date when people paid me, revenue was up 12% over 2018. If it’s based on when I did the work and how much of it was collected, then it’s down 1%. There are several hundred dollars owed me at the moment and one client that won’t seem to pay me the $45 owed. I guess I’ll have to write that off. Thankfully, it’s the first time in years that I’ve been stiffed. Sometimes I have to nudge clients gently, but they usually pay me. Most pay within hours of completing the work.
In any event, since I count taxable income based on the date of receipt, I guess it was a better year than 2018. In 2020, I expect to publish a book for forum administrators. Hopefully there will be an additional revenue stream from that, but it will come with expenses, principally paying an editor to clean up my grammar and typos. That’s going well and I’m about ready to turn it over for editing. I’m using LibreOffice and noticed a big quirk saving it as a .docx then moving it back, which messed up page styles. So I have to redo all that in a clean version of the book, which is a lot of reformatting.
Work-wise, December felt pretty average:
I updated a board from phpBB 3.2.5 to phpBB 3.2.8. I also updated the IDFresh Launch style from version 2.0.1 to 2.0.7 and updated the Google Analytics extension from version 1.0.3 to 1.0.4. But the extension would not install and directed me to login. So I installed it manually via the database and uploading files, although migration is missing. I tweaked stylesheet.css to make the logo 77 pixels wide. Client asked about some special reports, doesn’t look like phpBB contains the data needed to give it to him. Eventually I convinced him that he could use Google Analytics to get the report he wanted.
Conversion and rehosting. Converted phpBB 2.0.21 forum to phpBB 3.2.8 by downloading files, converting on my machine, and uploading to new host. Client moved board to a new domain name too. 136,000 posts. I changed the internal links in posts to point to the new domain using the database. I added the External Links in New Windows extension so they would have nofollow attributes. Made the AllanStyle SUBSILVER style the parent style and created a custom style, overriding logo change and moved the poster profile to the left side of the view topic page. Removed 1111 inactive users, likely all spammers, with no posts. Added AWS S3 extensions to allow attachments to be stored on AWS. Installed Google Analytics extension. Recreated search index using the MySQL fullindex, since it was faster. I created a reCaptcha V2 checkbox spambot countermeasure. I changed time for being online to 4 hours. I enabled quick reply in all forums. Need certificate to do https. www prefix not available. Set up redirect rule for old domain to redirect traffic to new domain.
Later did more work for the last client. Forced www to use non-www URL. Forced http traffic to use https. There was more work fixing links in the posts for some edge case URLs. Made user registrations go through an email verification process. Removed image links in signature lines for 12 users. Created a robust robots.txt file. Disabled contact form to reduce spam. Enabled active topics feature for all forums. Removed the website custom profile field. Set email settings to use SMTP which required a lot of testing. Changed some administrator permissions to remove founder status and reduce the administrator’s roles for these users. Increased moderator permissions to give moderators full moderator role privileges. Then there were more styling changes. Removed who’s online from the index. Moved Quicklinks and FAQ to a different place on the navigation bar. Removed the style’s credit line. Changed default font size from 10px to 11px so style does not lay out with such a dense look. Changed the forumlist_body.html template to remove moderators for forums on index. Fixed breadcrumb issue with the style. Replaced the logo with one a bit larger. Later, there was more work. I installed the sitemap.xml extension. I changed digests so users would get weekly digests instead of monthly. Changed forum permissions for a whole lot of groups making a number of assumptions.
Helped a client think through the issue of bringing in the content in a closing Yahoo group into phpBB. After much discussion, client decided not to import them into a forum/post/topic structure, but to make them available outside of phpBB and link to them with a sticky topic. So my role was to figure out what the client really wanted and provide advice.
GoDaddy brought down a forum’s connection for database, which we eventually learned was due to exceeding 1GB of database quota. To solve that problem, I used phpMyAdmin to truncate the search tables, which brought it under 1GB, then turned off the board’s search interface. I then had to attach the database user to the database. GoDaddy had disconnected it. I deleted spam chat tweets in mChat. I removed a few hundred inactive users with no posts. There is a problem with spam posts, probably because his Cleantalk subscription failed silently and it did not get renewed. Those need to be removed, so I waited on him to pay Cleantalk to renew subscription. Updated the board from phpBB 3.2.2 to 3.2.8 and created a custom style based on prosilver that I enabled. Four extensions could be upgraded. Later, there was one more hour troubleshooting. I renewed the Cleantalk subscription for the client and added it to his bill. I backed up his posts table then used Cleantalk to remove spam. But there was still plenty of old spam left over. The issue seems to be that this Cleantalk feature does not check the post content itself for spam, but the IP of the poster when the post was made against its database and flags those. So if the IP address is no longer flagged as a spam IP address in their database, it gives it a pass. I did an analysis of posts per day and sent to client to help determine when spam started in earnest. Board looks largely infested with spam. If it could be rolled back to a time before spam, the content may be relevant again. I changed reCaptcha keys. Board is currently disabled. No automated way to clean up spam appears to exist. However, I did some investigation and it looks like an Akismet API could be used if I wrote a program to send the API the post text, IP of poster and post type. Waiting to see if the client will pay for this.
Assisted in rehosting a board and a site. Provided advice on new hosts. The files on old host were a mess, a lot of it (like the Joomla front end) weren’t to be used on the new hosting. The client had already transferred the domain so things weren’t hooked up. Moved forum files and database to new host. Also moved files only in public_html folder because a lot were references to these files in the forum. I reconfigured an ad on front page to fix its links. I made everything use https.I redirected the domain to forum folder. I set up reCaptcha V2 spambot countermeasure, and disabled contact form. I updated the American English language pack but there are missing strings. Then there was a lot of hassle getting the email settings working right. I had to set up an account for SMTP and puzzle through the settings. My digests extension was returning errors, so I upgraded it from version 3.2.3 to 3.2.16.
There were error messages on forum index. The issue was that PHP 7.3 was being used. In cPanel, changed PHP to 7.2 for the domain. They came back, so I reset PHP version back to 7.2. Later error occurred while PHP 7.2 was installed. Commented out an echo statement in /includes/functions.php made the error go away. The overlying issue may be he needs to upgrade phpBB. Moving PHP back to 5.6 seemed to make all these issues go away.
If you’ve been waiting for phpBB 3.3, you’re not alone. The phpBB Group is famous for not setting release dates. They arrive when they arrive. There is no big company like Automattic is for WordPress behind phpBB.
I took a development version of 3.3 (3.3.0-RC2-dev) for a spin today to see what’s new. And the answer is not too much, at least so far. There are only two things I noticed right away:
The phpBB logo is now a Scalable Vector Graphic. This means it will scale nicely and always look sharp and fresh.
When you access the General tab in the Administration Control Panel, statistics are laid out differently and are arguably a bit confusing. Two tables with statistics? The first block contains counts. The second one contains metadata. It could be better laid out
What phpBB 3.3 “Proteus” really seems to be about is keeping it up to date. So far I don’t see any new features. A one-click update is supposed to be in 3.3, but I don’t see it. I suspect it got deferred for phpBB 4.0.
How is it keeping it up to date?
PHP 7.3 and 7.4 will be supported. These versions of PHP have been out for a while, but phpBB wouldn’t work reliably with them.
PHP 5.6 and 7.0 will not be supported. The minimum supported version of PHP will be 7.1.
The reCAPTCHA Invisible CAPTCHA will be supported. Currently, only V2 Checkbox is supported. This means CAPTCHA can happen implicitly. You shouldn’t need to click on a checkbox to prove you are a human.
Two new password hashing algorithms will be allowed: Arg2i and Arg2id. Passwords are not stored in the database in plain text. But using these newer algorithms, passwords become harder to crack. The 2id version is also much faster than the 2i version.
phpBB is dropping support versions of IE below IE11. That’s because the versions of Windows these run on are no longer supported by Microsoft, so they feel free to let them go
As a consequence of the above, jQuery 3.4 will be included. This version works with newer versions of browsers and pointedly does not support ancient versions like IE7.
If there are other new features, I’m not finding them. For most users, except for the phpBB logo, it will look and behave as it did under 3.2.
This was a pretty busy month overall. I did not have time to work on my extensions, although two extensions went through a formal review (Digests and Selective mass emails) but were, alas, rejected for more code changes. I am working on the book on phpBB administration and nearly done with a first draft. I hope to get it professionally edited but it needs a lot of work.
Updated board from phpBB 3.2.5 to phpBB 3.2.8. I also updated the Cleantalk extension from version 4.6.9 to 4.7.1. I reapplied the logo. The forum uses the default prosilver style.
A client getting hammered by bots in China and probably Hong Kong too. He was experiencing lots of HTTP 508 (loop detected) errors. I installed by Filter by Country extension (beta) 1.0.8 to see if that helps, blocking IPs from both China and Hong Kong. Analyzed Awstats statistics for October that indicated a lot of Hong Kong access. Client also blocked the Hong Kong range of IPs in cPanel. I changed the spambot countermeasure to reCaptcha. I disabled the contact form. More work may be needed later depending on how well this works.
Troubleshooting. A client had an issue with “Secure connection failed” error messages. I advised to change server settings, change protocol to https and port to 443. Later, I gave advice to use the domain name instead of an IP in the server settings. This was peculiar in part because the client wouldn’t give me access to his forum, so I couldn’t directly see the problem. Also, for payment he mailed me cash!
Client was having an open_basedir error with his Joomla content management system. Although not my area of expertise, I worked on it. The /logs and /tmp paths in Joomla’s configuration.php were incorrect and had to be pointed inside the Joomla folder. I figured out the absolute path and made these changes. The issue, or at least the reported error, seems to have gone away.
I updated a board from phpBB 3.2.7 to phpBB 3.2.8. I updated the board’s prosilver_se style to latest version, as well as Danish and German casual honorifics language packs.
The previous board owner has been non-responsive. He may be dead, and board issues were piling up. Traded some emails with an alternate administrator with no founder privileges on what I could do for her. I moved some users between groups. I also made her a founder. I answered questions on how to change group color, and ranks vs. groups.
Client failed trying to upgrade a board from phpBB 3.0.8 to phpBB 3.2.7. All files in the board’s root folder were missing including config.php, but the folders were there except for files, images and store folders, which I uploaded. I uploaded missing files too and created a config.php file, after finding the database and creating a database user. I then could update the board. Attached images in posts may be missing, along with avatars, ranks, etc. I changed folder permissions as necessary. Later, ran the update to phpBB 3.2.8. There were no issues after I corrected the username to use. I disabled the contact form. I did not change their spambot countermeasure.
I updated a board from phpBB 3.0.14 to phpBB 3.2.8. There were no mods on the old board, but AutoMOD was installed that I cleaned up manually. About 180,000 posts. No issues during upgrade but entering the ACP caused a VigLink error that I fixed by removing a row from the phpbb_config table. I installed the proprietary Milk style, which also meant installing its extension first. I gave the style a default color so I could see it, otherwise it was washed out. I disabled the contact form and added reCaptcha V2 checkbox spambot countermeasure. I changed the board’s config.php file to use mysqli. I recommended upgrading PHP to 7.2.
I converted a forum from phpBB 2.0.22 to 3.2.8. About 120,000 posts. I kept the default prosilver style for now. I installed the German Casual Honorifics language pack. I installed the GDPR Extension (Release candidate) and Advertisement Management extension. I disabled contact form and recreated the search index. I set up reCaptcha V2 spambot countermeasure. Client watched the whole process online with Skype. I suspect he will do the fine tuning.
As the name implies, forum roles control the privileges to forums. Forums are the key structure in phpBB and where most of the action happens, so there must be a lot of ways to finely tune access to forums.
Forum roles are most typically used to control forum privileges as they simplify the process. I will also demonstrate a way of circumventing these roles to set more granular forum permissions.
User roles provide a broad set of permissions, many of which extend to work users do in forums. Allowing attachments to posts is one example. Since roles are bundles of permissions, permissions in forum roles may override some user role permissions.
Pre-defined forum roles
The following forum roles come built in to phpBB:
No Access. Can neither see nor access the forum. This should be applied when you want to hide a forum from appropriate groups and users. You most typically use it to hide forums from guests and bots.
Read Only Access. Can read the forum, but cannot create new topics or reply to posts. You often see this role applied to guests.
Limited Access. Can use some forum features, but cannot attach files or use post icons. This role is often applied to newly registered users.
Limited Access + Polls. As per Limited Access but can also create polls. This role is also often applied to newly registered users.
Standard Access. Can use most forum features including attachments and deleting own topics, but cannot lock own topics, and cannot create polls.
Standard Access + Polls. Like Standard Access but can also create polls.
Full Access. Can use all forum features, including posting of announcements and stickies. Can also ignore the flood limit. Not recommended for normal users. This is often applied to more privileged users such as moderators and administrators.
On Moderation Queue. Can use most forum features including attachments, but posts and topics need to be approved by a moderator. This role can be applied to a problematic poster known for making inflammatory posts.
Bot Access. This role is recommended for bots and search spiders. It does allow bots to read the forum, so if you don’t want bots to read the forum, the bots groups should use the No Access role.
Newly Registered User Access. A role for members of the special newly registered users group; contains NEVER permissions to lock features for new users. This gets around a quirk in phpBB where newly registered users can start new topics (which have to go through moderation) only because they are also in the registered users group. It’s strange that phpBB is not configured this way by default.
There is one other implied role: No role assigned. As the name implies, it indicates a lack of permissions in the defined context, so other forum permissions if they exist are used instead.
Setting forum roles
Generally, it’s best to attach roles to groups. Attaching roles to users can be done, but it makes it hard to fix permissions issues. Most likely, roles already exist for groups accessing your forums. However, when you create new forums, you often need to attach some roles to the users that will use them, generally through group forum permissions: ACP > Forums > Forum based permissions > Group forum permissions.
First, select the user group whose roles you want to change or add. In the Look up usergroup dropdown, select the group then press Submit.
On the next screen, select the forum or forums you want to add or change roles for, in the appropriate Select a forum select list or dropdown. The dropdown is used to select a set of forums inside a category. Then press Submit.
Finally, you have an interface for changing or adding roles for each forum for the user group selected. See screenshot below. In the Role dropdown, select the role to be applied for the forum and user group. When all are set as desired, press the Apply all permissions button at the bottom of the screen.
Creating new forum roles
You can define a forum role using similar procedures for user, moderator and admin roles: ACP > Permissions > Permission roles > Forum roles > Create role. In general, the existing roles make it unlikely that you will need to create other forum roles.
Overriding role permissions
While not a good idea generally, I should point out you can override forum role permissions for groups and users. Use either:
ACP > Users and groups > Users > User forum permissions
ACP > Users and groups > Groups > Group forum permissions
Here’s an example of how it can be done for a user’s forum permissions.
In this case, Jane Doe is a teacher and in the teacher’s group, so she has Standard Access role’s permissions to the teachers’ forums. This means she cannot post sticky topics, i.e. posts that stick to near the top of the list of topics in a forum.
We would like to allow her to post stickies but not change her permissions otherwise.
We first enter her name by entering it in the Find a member field and pressing Submit. See screenshot above.
Then we pick the forums where we want the permissions applied. In this case, it makes sense to select the Teachers forums category and the forums inside it. Once selected, press Submit. See screenshot above.
On the next screen, the role shows no role assigned because no user role has been applied. The user still has a group role applied. Click on the Advanced permissions link.
You can then select any permissions you want to grant. In this case, I granted the Can post stickies permission by changing it to Yes. Since multiple forums should be show on the screen, do this for each forum in the category. See screenshot below, which shows only the permissions for the category.
Clicking the Apply permissions button will make the permission stick for this forum, or do all then press the Apply all permissions button at the bottom. Now Jane Doe has the necessary added permission, but it was done outside of the forum role.