This month was pretty busy with work, but also distractions. One primary distraction was trying to figure out why some people were not getting digests from my digests extension. A user was kind enough to give me access to a test board where I could troubleshoot the process. Over many days I discovered the issue was due to changing PHP “continue” statements to “continue 2” statements in the last release. I removed this code because it doesn’t seem to be necessary. Its purpose was to prevent duplicate digest from going out, which users had reported. But with a phpBB or system cron, it shouldn’t matter. The cron logic itself should not allow this to occur for these types of crons. Removing it seemed to fix the problem. I hope to get out a new release of digests in the next few days.
I’ve also been working on a book with the title Mastering phpBB Administration. It finished the editing process late this month. Now it needs cover art, possibly some reformatting for the printed page, then publication and marketing. I expect it will be mostly downloaded as an eBook. I’m hoping I can do this in April. I’ve invested several thousand dollars so far and I’d like to start seeing a return on investment. It’s my first attempt to write a book, and I can appreciate all the work it takes to do it right now.
- I upgraded a board from phpBB 3.1.9 to 3.3.0. Since I last worked on it in 2016, it was moved to a subdomain. I had to upgrade the version of PHP since at least version 7.1.3 was needed and version 5.6 was present. I updated the German traditional and German casual language packs. To address an umlaut issue, I changed the collation for phpbb_posts.post_text column to utf8_bin and created sample topic to verify it fixed the issue in new posts. I then fixed issues with umlauts in many tables using the MySQL replace function.
- I placed an ad to appear at the top of pages using the already installed advertisement management extension. It took some trial and error to get the CSS right so that it centered properly. Later, I fixed an issue with various WordPress blogs on the site. The /wp-content/uploads folder permissions were 755, changed to 777. There is still an issue related to updating these blogs that I am still working on. Certain permissions are missing.
- I upgraded two boards for the same client to phpBB 3.3.0 from 3.2.1. On first board, I updated the prosilver_se style and two German language packs, then made a custom style to ease future maintenance by encapsulating the unique changes in the custom style. I also disabled the contact form and installed a set of Invisible reCAPTCHA keys. On the second board, I updated the Latte style, reapplied minor changes to .sccs files in one template then recompiled the style. I also updated my digests extension from 3.2.6-RC13 to 3.3.0.
- I upgraded a board from phpBB 3.0.14 to 3.3.0. To do it, I had to remove a number of dead extensions and modules from the database. I moved over digests subscribers from the digests mod. I tested a new cron job used to kick off digests, but the manual mailer did not seem to work properly. I installed the ACP Add User extension. I disabled the contact page. I created a reCaptcha V2 Invisible spambot countermeasure. There were AJAX issues installing extensions related to a CORS error. After doing it enough time, the extensions seemed to install correctly. Later, the client had a script that used data in the user_interests column which was not copied over. I populated it from my backup, but the script should reference the phpbb_profile_fields_data table instead, where the data now resides. Client was concerned about & that appeared as text in digests. I removed a line of code from my digests extension that added this entity as it no longer triggers a validation error.
- I upgraded a board from phpBB 3.2.8 to 3.3.0. I had to disable modSecurity in the .htaccess file to get into board after upgrade. Updated the advertisement management, Cleantalk, Board 3 portal and Tapatalk extensions. Tapatalk would not work on 3.3 without upgrading to Beta version 2.1.7. Changed the board’s .htaccess to redirect to portal by default. I reapplied the logo. i changed the spambot countermeasure to reCaptcha V2 Invisible.
- Client had issues with a custom style created by another developer when his board was upgraded to phpBB 3.3.0. The simple solution is to copy prosilver’s templates to the custom style and purge the cache. Instead, I made a proper custom style that inherited from prosilver. Only the CSS changes needed to be applied to prosilver are now in the custom style, making it easier to upgrade the board in the future.
- I upgraded a board from phpBB 3.2.7 to 3.3.0. Client had issues doing it himself using the automatic upgrade method, which should not be used for phpBB upgrades, but only updates.I had to change PHP to version 7.1. A redirection issue on main .htaccess file for site meant I had to rename it temporarily to do the install. Changed reCAPTCHA spambot countermeasure to use the invisible CAPTCHA. Had to update Cleantalk extension to 5.7.2 to get it to install.
- I do ad hoc work for an existing client as user issues come up. In March:
- I changed an email for a user to her gmail address. Later, I heard there was another problem with her access, but when contacted she said she was getting email notifications.
- I informed another user of her username and provided lost password instructions.
- I forced reactivation for another user who could not login. It didn’t seem to work, so worked over the phone to get him online.
- Changed email address for another user.
- Provided instructions for a user to get a new password.
- Advised the board’s administrators on a DNS issue sent by their hosting provider.
- Provided lost password assistance to another user.
- Changed the email address for a user and provided resetting login procedures.
- A user wasn’t getting email notifications. Asked her to look in her spam folder and see if they are there. Eventually figured out her email address was incorrect for receiving notifications and changed it.
- I created a new password for a user so he could login.
- This month in a long term project I am doing for a client, I was trying to do a test upgrade of their phpBB 3.0.5 database to phpBB 3.3. My first job is to update the style to match their current style. Both attempts failed:
- In the first attempt, I realized that I needed PHP 5.6 to install the support toolkit to remove bad data from the database. I tried to install it myself on their machine but not being a system administrator, it didn’t seem to work.
- I next downloaded their database and tried to convert it on my laptop on which I had installed PHP 5.6. The database eventually imported (four hours later) but the laptop is underpowered. phpBB’s support toolkit flaked out when I attempted to use it. An attempt to run the upgrade program didn’t work very well. The client made lots of changes to the database that had to be cleaned up, but it would still not upgrade properly. PHP 5.6 was not the best choice. I really needed PHP 5.2, since trying to upgrade I had to work around issues making phpBB 3.0 work with PHP 5.6. It’s getting harder to set up test environments that use older versions of PHP.
- I then imported the database into a virtual machine on my Mac, and a reference version of phpBB 3.0.5 as well so I could use its database to compare with the client’s. I realized I needed to understand all the changes made to the database, because I may need to leave data in the database because it might be needed when moved to phpBB 3.3. I have a call with the client today to look over these changes and factor them into my next test upgrade attempt.
I’ll be on a JoCo Cruise and thus unavailable. In fact, I probably won’t be online during this period.
This was a fairly ordinary month, but I have started two long-term projects, details of which are below. Both should keep the business comfortably afloat for much of the year.
Work on my book Mastering phpBB Administration continues. It is still with an editor but should complete editing within a few weeks. It still needs cover art, reformatting, and setting up a publish on demand mechanism. I’ll probably offer it as both an eBook and a printed book. Existing clients will be entitled to a discount. Figuring out the pricing will be challenging. At over 300 pages, it’s a lot of information. Publish on demand allows both color and black and white copies to be requested, but color copies are much more expensive to produce, so I may offer both at different price points. Price will probably not be less than $19.95 a copy. Stay tuned for more details.
- Began an exploration on whether a Delphi forum could be converted to phpBB. The client provided a set of .yaml files with the board’s posts, which also includes some images that were in the posts. My basic approach is to see if I can get the data into a format that can be converted into the old phpBB 2 database format successfully. Is so, then it could be converted to phpBB 3.3, the ultimate goal. The first step was to try to find a good YAML parser. I tried a number of approaches but eventually I used Symfony’s yaml parser in my script and that seemed to work reliably. The .yaml files were not perfect YAML, so I had to do a bit of editing to get files to parse correctly. I parsed the first 500 posts and placed the result in a HTML table with a script, then did the same thing with a .yaml file with the user information. These were sent to the client. The client will discuss with his group whether he wants to fund further research, but it looks likely because it looks like there is enough data in these files to populate a phpBB 2 database. He will get back to me in a couple of weeks.
- A client was running out of quota and asked me to figure out how to solve the issue. 4.9GB out of 5GB was in use. Problem was WordPress backups in a backup folder. Suggested removing old backups. Once identified, client took care of issue.
- After large upgrade to phpBB by a client, the modules for my digests extension got lost. On Skype, I walked him through the process of creating both the ACP and UCP digest modules manually.
- Upgraded and rehosted board from phpBB 3.0.5 to phpBB 3.3.0. The new hosting is on Siteground. The old hosting was on a Windows server. Fortunately, MySQL was used on the Windows hosting, so there was no database to be converted. There were some technical issues on both hosts with downloading databases and making archives. Siteground’s new site tools turned out to be confusing as cPanel was not an option. I could not grant permissions to the database user I created to database I created. It took two tech support calls but eventually it all worked. I created a custom style to show logo, otherwise default prosilver style was used. I disabled the contact page and created reCaptcha V2 Invisible spambot countermeasure.
- I provide ongoing support for a client’s business board. I investigated an emailing issue where emails were getting flagged as likely spam when sent to wvu.edu. The client’s domain is not on an email blacklist. This was something on wvu.edu’s end and the easiest way to deal with the problem was to have change the board user’s email address. I provided instructions on how to change email address used by the board. At the end of the month I updated two packages in Plesk and updated Plesk to version 18.0.24. Also, changed one user’s email address. She was not getting email notifications because the address had changed and she didn’t know how to change it in phpBB.
- I restored a lost board. Originally it looked like a backup of phpBB 3.0.11 was available, but it wouldn’t upgrade since some tables were in a phpBB 2 format and some not. The client found an older phpBB 2 database backup and I was able to convert that offline to phpBB 3.3. In the first conversion, the attachments were missing so I had to do it over again. I also changed the board to use to https. A few minor errors were encountered. I had to add an auto increment to the phpbb_logs table’s primary key. I set up reCAPTCHA and disabled contact page.
- Updated forum from phpBB 3.2.2 to 3.3.0. I had to comment out Options -MultiViews in .htaccess file to avoid HTTP error. Also had to remove install_config.php file in the store folder to get the upgrade to start. Otherwise there were no issues. User had warnings that were due to PHP 7.3 being enabled on his host, which is not supported by phpBB 3.2.
- Began work for a new long-term client who has a large job to upgrade his phpBB 3.0.11 board to phpBB 3.3. This will take many months as there are many customizations embedded in the source code itself that need to be turned into extensions. This month I mostly looked through their changes to phpBB 3.0.11 and advised on the best approach. The first step will be to try to change the style to work with phpBB 3.3. The style uses SCCS and needs to link in a lot of CSS and HTML used for the master website. I may develop some of the extensions myself of work with other extension authors to have some develop them. Some look very complex, such as a proprietary OAuth integration. But it’s commercial work and pays very well, so it should be both challenging and financially rewarding. More on this in coming months.
phpBB is great but not perfect forum software. phpBB has many upsides, but it has downsides as well. In the interest of fairness, I thought I would document some of them.
These will also appear in my Mastering phpBB Administration book, which is undergoing professional editing at the moment, so consider this a bit of a preview. I’m hoping in a month or so it will be published. Here’s an except from the draft of the book on its strengths and weaknesses, as I see them:
- Longevity. It’s been around since 2000.
- Market share. According to the phpBB Group, phpBB is the #1 bulletin board solution in use. If you type “bulletin board software” into a search engine, generally phpBB is at the top.
- Support. The phpBB Group’s support forums are outstanding. Responses to questions are quick and almost always helpful.
- Security. While no software can guarantee it is free of security vulnerabilities, phpBB’s combination of open source software, code peer reviews and their extensive automated nightly build tests minimize unexpected security issues.
- Rock solid. While most phpBB boards are relatively small, with just tens of thousands of posts, some are huge with millions of posts. As of this writing, phpbb.com’s forums alone have over 4.2 million posts. It needs to just work and almost all the time it does, at least if you use the latest version of phpBB, stick with approved extensions and your web host’s technical infrastructure doesn’t change too quickly. When phpBB fails, it’s almost always due to hosting changes beyond its control.
- Features. If you need a feature, it’s likely available in phpBB, with one major exception (see the next section.)
- Familiarity. Since you have likely used it before without knowing it, there is no steep learning curve.
- Fanatical devotion to open standards. The phpBB Group goes to great lengths to ensure phpBB works across all browsers and devices. It does this by carefully adhering to the latest web standards and daily automated testing of builds in development.
- Responsive. It behaves seamlessly on mobile devices, intelligently sizing down to the device’s screen size, yet with no loss of functionality.
- Uses top-tier integrated third-party libraries. Under the hood, phpBB uses a host of other top-notch, enterprise-class software libraries. For example, it uses a Twig templating library from Symfony for rendering web pages with dynamic content.
- Supports lots of databases. Typically phpBB is used with the MySQL or MariaDB database management systems (DBMS). But if you want to run it on the Oracle, Postgres, Microsoft SQL Server or even the SQLite DBMS, it will work and function virtually identically.
- Permissions system. There is probably no better permissions system available anywhere. It’s incredibly feature rich, if more than a bit obscure.
- Extensions. Since version 3.1, phpBB supports extensions. Extensions are new features that you can add to phpBB that can be turned on or off once you install them, all without affecting its base code. Extensions are not currently quite as easy to use as WordPress plugins, but they are getting there. And the number of approved extensions just increases with time.
- It’s maintained and updated. If a security issue is uncovered, it tends to get fixed promptly with patch instructions to use until there is a new release. Over time, new releases will update phpBB so that it works with the latest changes to technology.
- No multi-threading. While a poster can quote from a previous post inside of a topic, you cannot see a group of related, indented replies to a post within a topic. Hopefully this will show up as a feature one of these days.
- New features are added slowly, if at all. While phpBB is a rock-solid bulletin board solution, it is not easily changed. This is in part because it is so feature-rich. Features that are added tend to be relatively minor and incremental. Rarely do you see big, new features. The extensions system introduced in phpBB 3.1 was one of these rare and big changes to phpBB.
- Standalone. phpBB doesn’t integrate with other software solutions. For example, you can’t integrate it into WordPress or Joomla, at least not elegantly. It’s not available as a WordPress plugin.
- Updates and upgrades can be painful. While better than it was, updating and upgrading phpBB is often a challenge. Most of my consulting business involves helping clients on these issues. Web hosting limitations also can introduce problems during these time-critical activities. For many users, the improved update process in phpBB 3.3 will do a lot to address this.
- Setting up a bulletin board is mysterious and somewhat painful too. This is one of the major reasons I wrote this book. Mostly, bulletin board administrators learn by doing. It’s so much better to do things the phpBB way … except you can get a hundred different opinions on what the best way is. I’ll risk the wrath of the phpBB community (they are a passionate, but very helpful bunch) by telling you what I think the best way is, and why. I will help step you through these phpBB mysteries based on more than a decade of practical experience.
- Complicated to administer. While using a phpBB bulletin board is not complicated and usually intuitive, administering a phpBB bulletin board can be very complex. With so many features, it’s hard just to know what they all are or where to find them. Certain very powerful features, like its granular permissions system and its ability to bundle permissions into roles are totally awesome, but rarely delved into. Sometimes an administrator won’t even know a feature exists.
So know what you are getting into. If any of these are major issues, you might want to use another bulletin board solution, or just not bother to install phpBB in the first place.
I did plenty of work in January, just not so much for clients. That business was slow, but picked up toward the end of the month, so much so that at the moment it looks like my plate in February should be full.
Why was I busy? I was writing and editing a book on phpBB administration. The introduction of phpBB 3.3 Proteus on January 6th added work, as I needed to update the book to talk about phpBB 3.3. As I noted, its feature set makes it more of a micro release than a minor release, so that part was good. But by changing the phpBB logo and a few other things, I had to recapture fifty or more screenshots, which was tedious and time consuming. Then of course the text had to be revised in places which also meant reading it through twice more. Right now it stands at 305 pages on 8.5 x 11 inch paper.
I reached a milestone of sorts when I turned it over for editing on Thursday. I should mention that I am using Maria Williams, so if you need an editor, hit her up! It will take her a couple of weeks for her to do her magic, then I have to figure out how to publish and market the book. I need cover art. Publishing will likely be on demand, and I expect most people will want electronic copies. I also need to figure out how to price it. It’s a sizable capital investment of time and money, so it needs to be profitable. I expect to offer some sort of discount for the first ninety days and for my existing clients. The last book published on the topic was written in 2006, so it described phpBB 2, so the market is out there. Also, that was a user guide, not an administrator’s guide.
- Added two users and forced a reactivation of two other users for a commercial client, who outsources this work to me.
- Updated a board from phpBB 3.2.7 to 3.2.9. Also, I created a new forum with a forum logo on the board. I resized and placed a forum image logo with the forum, and put it in the requested place on the index. I gave the forum permissions similar to another forum. I investigated issues in the error log. Most of these errors seems to be due to debug being turned on.
- More analysis for a client I did work for in December. I generated a list of users and email addresses on his archived forum that had contributed at least 1 post but whose usernames were not on the current forum. He will use the list to invite them to come to the new board. I installed a release candidate extension to creating lots of user accounts in bulk, but it didn’t work as advertised. I did some styling changes. I upped base font 1 pixel to 12pixels. Client wanted to know if my digests extension can show text only. Yes, and I sent him a sample. He wants digests generated by my extension to look similar to someone else’s digest. I deleted a user defined moderator group. I provided guidance on using the ACP Add User extension so lots of old users could be added. He would like to delegate the work to someone else, so discussed how this could be done with ACP permissions so their access was tightly limited in the ACP.
- Client’s board wouldn’t come up on Bluehost hosting. After some analysis, I determined he had a phpBB 2.0.19 board and that only PHP 7 was available. He did have attachments. I converted the board to phpBB 3.3.0. Issues logging on were solved by removing the domain’s cookies, clearing browser cache and resetting the password, which was set back. I moved the old board into forum_old folder. I created the search index, disabled contact page, added Invisible reCAPTCHA. The resulting board is very basic. It may need some customization, styles and extensions installed. Later there was additional work. The client needed some gallery software. The phpBB Gallery extension is not being maintained and is not reliable, so looked for a solution outside of phpBB. I tried one solution, then tried another. Piwigo seemed to work after I installed a Community plugin/extension. This allowed users to upload their own pictures.
- I updated a board from phpBB 3.2.0 to 3.2.9 on generic prosilver style. I updated my digests extension to version 3.2.17 and the Cleantalk extension to version 5.7.2.
- I upgrade board from phpBB 3.2.7 to 3.3.0. I updated the prosilver SE style to 3.2.9 and put the old logo back. I changed the spambot countermeasure to reCaptcha V2 invisible, since spam registrations were a complaint.
- Client wanted active topics in red on the navigation bar. First tried installing the Advanced Active Topics extension (release candidate) but couldn’t get it to show. I then installed the Recent Topics extension as a trial, but client didn’t like it, so I created a special link on the navigation bar, highlighted in red, moving it from the Quick Links menu.
- I upgraded a board from phpBB 3.2.8 to 3.3.0. I replaced the logo.I checked all the extensions and they seemed to work on phpBB 3.3 with no issues, but there was no test case for change post time extension to verify it worked properly.
- More work for a client on spam. Used Prune Users feature to remove users and their posts who registered after Feb 10, 2018 when spam seemed to start in earnest. Reenabled board and suggested ways to mitigate the problem in the future with moderation, digests, etc. This is discussed in more detail in this blog post.
- This work is still ongoing. On Hostgator hosting, I created space for five domains and set up DNS for these domains. I set up five boards on five subdomains of these domains. All have a basic configuration, registration disabled, reCaptcha enabled, prosilver_se style applied, contact form disabled, and forwarder set up to redirect to client’s email address. Set up https redirection by installed WordPress SSL plugin on the main blog. Next, I cloned four WordPress instances based on work on one domain, whose theme and plugins client wanted to replicate. I did this by copying the files and database, changing the wp-config.php file to point to the new database, and tweaking the site information in the wp_options table. This saved a lot of time and work for the client.
Only minor changes were made, but it is tested on phpBB 3.3 and works on that and phpBB 3.2. Details here.
It can be downloaded from the GitHub branch or from the extension’s page. If downloaded from GitHub, make sure to disable the extension first, remove the old filed and place the files in the /ext/phpbbservices/selectivemassemails directory.
Announcement is here. This version works on phpBB 3.3, although it required no change in functionality, just some changes to assert that it would. There were no language file changes.
If you download the extension from the GitHub branch, make sure to disable the extension, delete the files, and upload the new files to the /ext/phpbbservices/digests extension.
So I’m starting to port my extensions to phpBB 3.3.
I’m surprised by how easy it has been, at least compared with porting my extensions from phpBB 3.1 to 3.2. This is because not much has changed between 3.2 and 3.3.
In moving from 3.1 to 3.2, there were all sorts of changes under the hood that made porting extensions challenging. The way posts were encoded in the phpbb_posts table was changed, which was a big deal. FontAwesome support, which was very limited, was introduced, meaning we could use these in our templates instead of embedding images. A new TWIG library meant that the old way we extension authors inserted dynamic data into pages changed pretty fundamentally. We were asked to use a new template syntax in our templates. Template library calls also changed quite a bit. In short, 3.1 to 3.2 required quite a bit of reengineering, both of the phpBB base code but also of its extensions.
For 3.3, there’s not much upgrading of the database needed. Only one table is altered: phpbb_users, to change the properties of the reset token. The content in the phpbb_extension_groups table is changed. The Flash media group finally goes away (Flash is obsolete) and any non-Flash files in the group are now assigned to the downloadable files group.
As noted, you can use more Emoji codes in 3.3 and can put Emoji in a topic title. But this doesn’t alter the database. I have to test for these in my extensions, but so far it’s not a problem because the existing function I used to display these fields handles it for me. So nothing to do there.
So phpBB 3.3 feels more like an update to phpBB 3.2 than it does a new minor version. On the whole, the new functionality is quite minor. The main thing I have to do is test with PHP 7.4, since phpBB 3.3 now supports PHP 7.1.3 to 7.4 only. Getting my extensions to a stable 3.2 place before upgrading PHP is turning out to be much more work than retrofitting my extensions.
So yea for that.
A new version of this extension that supports phpBB 3.3 is available. See this post for details. It works with both phpBB 3.2 and 3.3. One bug that was writing double statistics was fixed.