August 2020 work summary

The economy may be hurting but not my personal economy. Demand for my services was high in August, so high it was hard keeping up with all the work coming in. It slowed a little toward the end of the month giving me a bit of a breather.

  • Long, painful, tedious conversion from a phpBB 2.0.19 board to phpBB 3.3.0, with a rehosting to boot. This project took over 3 days. 3.9M posts were converted. The board used the standard prosilver style. (phpBB 2 is so old there was no prosilver style back then. phpBB 2 was obsolete in 2007!) Customized and centered the logo. The board has 287,000 users, mostly spammers. Installed advertisement management and search on navigation bar extensions. Showed client how to place sample ad. Installed Italian language pack. Created a custom style to make future upgrades and updates easier. Had to optimize tables due to slow performance. Database upgrade took about 30 hours and timed out once. Very tedious, time consuming and not very profitable work. Created a new MySQL fulltext search index.
  • Upgraded board from phpBB 3.2.8 to 3.3.0. Installed updated Danish and German casual language packs. Updated the NavBar Search extension to version 3.2.4 and the Tapatalk extension to version 2.1.8. Put old logo back on prosilver_se style, which was also updated to a 3.3.0 version. Changed spambot countermeasure to use reCaptcha V2 invisible. Later, updated client to phpBB 3.3.1 at no extra charge since it was just released.
  • Client wanted to rehost board on new server. On the new host the database had already been moved (presumably by the new web host) but the files had not. So I made a phpBB files archive and moved them over. I then corrected config.php to use the correct database username and password, and changed the password to match the old password. Downgraded PHP to 7.2 since phpBB 3.2.7 was installed. The WordPress part of domain came up fine.
  • Upgraded a board from phpBB 3.0.13-PL1 to phpBB 3.3.1. Tried to replicate the prosilver_se style but had logo placement issues because part of the logo had black within it. Client will be redo the logo, but changes to the stylesheet happened erratically. I wondered if this might be due to a CloudFlare or CDN issue. Installed my latest version of my Smartfeed extension and Google Analytics extension. I changed PHP to 7.3. I disabled the contact form, and created a reCAPTCHA V3 spambot countermeasure. Client had a script that independently queried the number of posts, topics and users to feed to another system. These did not work with the phpBB upgrade. The easiest solution was to rewrite it to call the database directly with mysqli to get the values needed. The board has 134,000 posts. Later there was an issue where the search index was not being indexed. Recreated the index. But the root problem turned out to be a bad configuration value in the phpbb_config table which had to be escaped to get show the correct class namespace. With it, it could then use the correct search engine.
  • Upgrade and rehosting. Moved phpBB 3.2.2 board and database to new Siteground hosting. Lots of issues importing the data into the database (timeouts). Upgraded board to phpBB 3.3.1. Changed registration text to say not to use real name when registering. Installed Filter by Country extension. Let’s Encrypt security certificate could not be installed until domain is transferred. Transferred domain using the client’s registrar. Installed Advertisement management extension and placed sample ad. Placed Donate button on navigation bar pointing to non-existent donate.html on the web root. Later there was some troubleshooting. Newly registered user did not belong to any group and saw a no forums message. Added him to both Newly registered and Registered users group, advised to change permissions for newly registered users using roles. Client said the notification count was incorrect. Might have been due to new notifications being received. Set up the board’s email settings to use SMTP and created an email forwarder associated with the domain. Also created an admin email address. Added email server as an authorized server for sending email, although first attempt showed emails arriving in my spam folder.
  • Troubleshooting. Users were having sporadic logout issues. Changed a number of cookie and server settings. I also made the server generate https URLs and changed permissions to /cache/production and /cache/production/twig to 777, were 755.
  • Updated a client’s board from phpBB from 3.3.0 to 3.3.1. Updated three extensions: Advanced BBCode Box, Precise Similar Topics and Recent Topics to latest versions. Reapplied logo.
  • Upgraded a board from phpBB 3.0.12 to 3.3.1. 305,000 posts. Smooth upgrade with one minor issue: no active style. Changed to make prosilver the active style in database and then the upgrade was able to complete. Installed topic solved extension but it needs per forum configuration, so I sent the client instructions on how to do this. Installed Stop Forum Spam extension, which is currently a release candidate extension for 3.3. Created a custom style based on prosilver. A number of styling changes were desired. Work to replicate the top of the header template was suspended as I needed link to a stylesheet I didn’t have, so I removed that code and added a Home button for now, which turned out to satisfy the client. Hid the site description. Recreated the search index. Disabled the contact form. Installed reCaptcha V3 spambot countermeasure. Added Google Analytics extension. Had to copy over some images to the custom style. reCaptcha V3 has issues: it couldn’t register users and there were curse words on registering when it failed. Reverted to reCaptcha V2 invisible. The main issue turned out to be that all @gmail.com email addresses were banned, and the defined banned list reasons returned curse words because the gmail.com was a blocked domain. Also, there were some obsolete Moderator Control Panel modules that I had to remove by hand in the ACP. Apparently the database cleaner did not get them all. Installed the Ban Hammer extension.
  • Investigation for client to see if non phpBB applications break on upgrade to PHP 7.3. Found only one that broke: a Zencart integration that will need an updated version. Upgraded board from phpBB 3.1.10 to 3.3.1. prosilver_se style was used with customizations, so I created a custom style that inherited from prosilver_se. Board has 773,000 posts and 71,000 attachments. There was lots of hassle getting the styling right. One issue encountered during upgrade: a config_name already existed in the phpbb_config table. Removed the row from the table and the upgrade then was able to complete. Changed PHP to 7.3. Updated 16 extensions. Installed reCaptcha V3. The /files folder had the attachments nested in folders inside of folders. Don’t know why this was, but it predated the upgrade, and caused images not to show in posts. The client assumed the onerous chore of moving these files correctly for display. PHP Gallery was linked from the board and on the navigation bar. It came up fine on PHP 7.
  • Upgraded a generic board with no mods from phpBB 3.0.11 to 3.2.10. No PHP beyond 7.0 was available on the web host, so it limited him to 3.2.10. I chose to upgrade PHP to 5.6 (5.2 was installed) in case there was other software on the domain that would not work on PHP 7.0. I had to edit first line of config.php because it loaded a nonexistent antispam program. Disabled contact form and set spambot countermeasure to reCaptcha V2 invisible. Some folders did not have public write permissions set.
  • Update board from phpBB 3.3.0 to 3.3.1, and changed PHP version from 7.1 to 7.3.
  • Made about $60 in book royalties last month.
  • Continued work for a large commercial client looking to upgrade their board to phpBB 3.3 while maintaining their unique look and customizations from the current phpBB 3.0 board. This month worked on two extensions to replicate logic that was previously done with modifications. The first was to allow moderators and administrators to find topics which had no replies by moderators or administrators, using the existing search interface. This took quite a while to puzzle through, but I was eventually successful. The second is to email an email list if any entry is made to the admin log. After some discussion with the client, I added logic to allow them to not email certain notifications by manually adding them in a text area form field, as these will be too common and thus annoying. I started work on an extension to automatically lock topics more than 60 days old. Thinking about it overnight though I realized there probably was an extension to do this and there is, so I installed that instead. Looking through the remaining work, there should only be one more extension to do. This involves a complex connection between phpBB and the accounting system the client has, which determines who can post in the forum and creates accounts in phpBB. It will require some sort of custom Oauth solution because that’s what they had before. Since styling work was done in July, this may be the last big piece of development work. Doing a test conversion and working out kinks will then be possible. So hopefully this project will be finished by the end of the year. This is interesting work and it’s nice to have a client pay my commercial rate and pay very promptly once a month.

phpBB 3.3.1 release summary

phpBB 3.3.1 was released on August 7, the first update to version 3.3 since 3.3.0 was released in January. You can download it from the downloads page.

If you are still using phpBB 3.2, there’s an update (3.2.10) for it as well, mainly to address a security vulnerability. For more see this page.

Since a security issue was fixed, an update is recommended.

You can read the release highlights here.

There are three new features, something you don’t often see in an update:

  • reCAPTCHA Version 3 is supported as a spambot countermeasure. You need to select it as the new spambot countermeasure and get a new set of keys from the reCAPTCHA site to use it.
  • Profile field types can be enabled and disabled
  • Only one email notification goes out when a topic has had new replies and you are subscribed to a topic. To implement this meant one new table had to be added to the database: phpbb_notification_emails.

One minor feature of the update was due to me: I filed a ticket asking for the inclusion of the DuckDuckGo search bot in the list of known bots. Woo hoo!

Some other things fixed:

  • Full search text searching is faster if you use the PostreSQL database management system
  • Support for FontAwesome Version 5 has been withdrawn
  • You can use multiple emojis in text fields
  • The private messaging bug was fixed. Without the patch, you were unable to delete or mark private messages read in the User Control Panel folder view.
  • Reset password issue fixed. It could have resulted in PHP fatal errors being thrown.
  • A bug was fixed that caused a failure when Emoji was used in private message subject line.

More minor issues fixed:

  • A bunch of missing configuration variables were added if they did not exist.
  • Conversions from phpBB 2 sometimes resulted in an invalid hash.
  • Bug fixed that would not allow a group avatar to be deleted in the User Control Panel.
  • Overflow error when attachments exceeded 2.1GB on SQLServer has been fixed. I found this as a bug years ago.
  • Google+ has been removed as a profile field, which makes sense as Google no longer has a Google+ service.

phpBB 3.3 is supposed to have a new feature to make updates easier. So far I don’t see how it is any easier and the process looks unchanged. As I do more updates for clients, I will see if I can figure out what is new and if so post some revised guidance.

July 2020 work summary

Felt a little slow around here in July. Could have used more work but I could have also used a couple contracts I’m on to move more quickly. I spend a lot of time waiting for clients to make decisions.

But my selective mass emails extension got approved this month, and the filter by country extension was resubmitted for another review. My Digests extension also got some rework to address issues in an April review but it’s not an official release yet as I’m waiting on language file changes. Strange that most of these issues really don’t change any of the behavior or add any features. It’s all to address perceived inadequacies in the code by the phpBB extension review team — really nitpicky stuff. No one will ever accuse the phpBB extension review team of not being thorough.

As for paid work last month:

  • I did more work for a client whose work I thought was done in June. Mostly it involved a lot of analysis and back and forth conversations. Emailing issues were reported so I provided guidance on where the real issue may lie (web host emailing quotas or filtering probable spam from going out.) I fixed two issues where the from email addresses was incorrect in some proprietary scripts written years ago by someone else. I went back and forth with the client about transitioning the domain’s static to WordPress using a custom plugin, or fixing the dated software instead, which is not responsive. More conversations are needed on what if anything they want to do next to modernize their website and their business process. The board was updated and moved to a new domain last month.
  • Upgraded a board from phpBB 3.2.8 to phpBB 3.3.0. I updated the style and updated the Google Analytics extension too. The correct logo image was uploaded. I made a minor tweak to stylesheet to render it correctly. I change spambot countermeasure from reCaptcha checkbox to reCaptcha invisible. Provided a sample SQL query to get a list of recent topics ordered by most recent post time.
  • This turned into a big job. I upgraded a board from phpBB 3.2.7 to phpBB 3.3.0. I installed a newer version of the Cleantalk, BBCore and Fancy Lazy Topics extensions. The latter two were used with the proprietary (paid) Flatboots style. There were lots of issue with HTTP 401 errors logging in after the upgrade. I eventually discovered I needed to add a form key to a Flatboots template that was missing. They had status images for forums (like the no new posts icon) in a weird place: /phpbb/avatars. I had to recover these files and move them to their proper places in the /images folder and correct the links in the database. There ensued lots of additional work, such as an HTTP 401 error going into ACP, which appeared to be due to being logged in to the same admin account from various browsers. Eventually we ditched the Flatboots style and BBCore extension, went with the Breezy style. I placed the existing logo inside it to customer’s satisfaction. This made the breadcrumb menu work correctly. I created a custom style inheriting from Breezy style for portability. But the 401 issue never quite went away and still seems to pop up from time to time. A workaround is to delete the board cookies. I’ve never seen this happen on any other site, so I figure the issue must be hosting related and not phpBB. My guess is that phpBB sessions aren’t getting deleted when they should, perhaps due to security software messing with permission in the /cache folder. It was all pretty frustrating, for both me and the client!
  • Some years back for a client I created some simple PHP software that listed a set of events kept in a database on a web page. The events folder containing software I wrote to record solar events somehow got deleted. The client was able to find an old copy of the files. I uploaded it and it began working again. The database fortunately was still up to date.
  • A client’s security certificate expired. I installed the Let’s Encrypt Plesk extension and installed a Let’s Encrypt certificate for the domain. In the past they paid for a certificate. With Let’s Encrypt, the certificate should renew automatically ever 30 days indefinitely. I provided analysis on SSL certificates options. They may not be comfortable with a free certificate and want a branded certificate instead. Later, I upgraded their board from phpBB 3.2.8 to 3.3.0, which included updating five styles (with changes needed to each) and 5 extensions. There were a few issues after deployment. A mailing list extension I used and modified had a newer version. I installed that but it caused issue, so I reverted to the old version of the extension with my changes.
  • Painful upgrade from phpBB 3.0.12 (showing as 3.2.0-dev) to 3.3.0. Running the database cleaner did not fix all issues in the database. Even moving the database to my machine and cleaning it there had issues. Eventually with enough tricks and fixing database issues that were encountered while running the database upgrade script, I was able to complete upgrade, which included populating the modules table from a reference, adding some missing configuration table values and adding some missing columns to the phpbb_profile_fields table. I disabled the contact form. I created a reCaptcha V2 invisible spambot countermeasure. Installed these extensions: Topic Index, Reimg Image Resizer (beta), Thanks for the posts, Recent topics, mChat and Search on Index. Search on Index was used because otherwise it would sit on top of a wide forum logo, which I had to center. Added a donate button but placed it on the navigation bar. I had to resize and crop it to make it look acceptable. Created a custom style and made changes there. There followed additional work and lots of questions. Answered questions about why there were three periods (…) in long links. Client wanted these removed but there’s no practical way to do this as it’s built into phpBB. With a bit of help on phpbb.com I put the recent topics above the mChat window on the index — I had to change the extension’s code to attach some code tied to an event to a different event. There was lots of concern about the placement and order of topics in the Recent Topics extension. After much back and forth I realized what he really want to do was bump topics so they would go to the top of recent topics shown. Apparently he had injected some code in his phpBB 3.0 board to do this. There is no extension to do this in phpBB 3.3 so he’ll probably have to live with it or pay someone to create an extension to do this.
  • Got some royalties paid for my book at last, but it was a pittance. I will need to sell thousands of copies to make a profit. Why not buy a copy now?
  • Upgraded a client to phpBB 3.3 who I had helped two years ago. A number of issues were reported. The basic problem turned out to be that the database had been upgraded, but someone put old phpBB 3.2 software back on the system. Uploaded the correct files. Updated their Absolution style but created a “custom” custom style with its changes to Absolution. Upgraded Digests, and User Details extensions. I had to edit services.xml file for Advanced Polls extension to get it to work on phpBB 3.3. Set spambot countermeasure to reCAPTCHA V2 invisible. There were quota issues uploading with FTP, so I had to use File Manager instead. I installed the Google Analytics extension rather than add code to overall_footer.html. Just today I checked a reported issue with digests not going out. There is some evidence that digests were mailed twice for some hours, I think because the board was down for a couple hours while I upgraded it. I’m hoping by resetting the digest mailer the problem will go away.
  • Finished a months long styling portion of project for a major customer. We’re now moving on to the other issues, principally how to replicate functionality they have on phpBB 3.0 using extensions. Met with the client and did a lot of analysis, but client was distracted with other issues so didn’t do too much work on this during July. This month I’ll be doing some experimenting with extensions to see if I can replicate the functionality they injected directly into the old phpBB 3.0 base code. They want to do initial moderation using Akismet. There is an Akismet extension for phpBB so I analyzed how that will work for them. Akismet will flag the spam, their moderators will review it and based on whether it is spam or not the Akismet database will get updated. This will save them a lot of work that was previously done through a lot of proprietary code.

June 2020 work summary

June was somewhat leisurely. During the downtime I created a new version of my Filter by Country extension, needed because it got its first review by the phpBB extensions review team and of course flunked. It took four days of part-time work to create a new version that addresses their issues and to fix some bugs I found, plus to add a new test mode feature.

I also released a new edition of my book, mostly to fix grammatical errors.

Most of my work was for a particular client I’ve been working with for some months now. They have a heavily modded phpBB 3.0 board that needs to move to phpBB 3.3. For a few months I have been working through styling issues, and did more of that this month, as it got a review by their designer. But the pace of the work was pretty slow. I suspect in July the styling phase will end with the changes I made accepted, and I’ll move onto a new phase of integrating their various custom changes into phpBB 3.3, hopefully using extensions. Ideally, no changes to the base code will be needed.

Otherwise, I did this work for clients in June:

  • Upgraded a board from phpBB 3.2.5 to 3.3.0. Updated we_universal style, my digests extension and the advertisement management extension to latest versions as part of the upgrade. There was an issue with /cache/production permissions reverting to 755. Seems stable for now.
  • Upgraded a board from phpBB 3.2.5 to 3.3.0. PHP changed from 5.6 to 7.3. This temporarily exposed an iconv issue (extension not loaded) which meant I commented out the code in the library. When the module was added, I uncommented the commented out code. I changed the reCaptcha keys to invisible. I reapplied an active topics link to a special place on the navigation bar and reapplied logo. I warned the client that moving to PHP 7 might cause issues with other parts of the website, but they told me to go ahead. Unfortunately, it did cause issues. They had many specially written PHP scripts that used old mysql drivers in the code and performed a lot of workflow logic. They decided to move these scripts to another domain on their hosting where PHP 5.6 was still available. But this had lots of unintended consequences because it was not fully understood the work that would be involved. The databases and scripts had to be moved but wouldn’t work as is. I had to change the database settings, and found a lot of HTML was in the database itself, which required changing the database. The old database became inconsistent with the new database on the new hosting. So I had to analyze and fix those issues too. I think there may be one or two lingering issues, but these issues seem to be mostly behind them.
  • Upgrade and rehosting. Moved a board from a folder in one domain to another domain entirely. I also upgraded the board from phpBB 3.2.3 to 3.3.0. I created an index.html file to do redirect from the old domain. A regular redirect would not work. I disabled the contact form and re-created reCaptcha keys for V2 invisible, as V2 checkbox is not supported on phpBB 3.3. I also had to add the American English language pack and update the Cleantalk extension for these to work on phpBB 3.3. Had an additional issue with the client still getting spam emails through contact page. Enabled Cleantalk to check it but was probably from the old domain, where it was still enabled. Client disabled it there too and the problem went away.
  • Upgraded a board from 3.1.3 to 3.3.0. There was a Plesk PHP issue in that PHP 7.0 was the latest version installed, and I needed at least 7.1.3. Client eventually installed PHP 7.3 so I was able to do the upgrade to phpBB 3.3. It uses the Subway style. Couldn’t get the extension for the Subway style to work but the style came up. In the interim, I made prosilver the default so the board could be usable. During the upgrade, I experienced frequent timeouts, whch meant I had to finish it using SSH and phpBB’s CLI (command line) interface. I disabled the contact form and changed spambot countermeasure to reCaptcha V2 Invisible. I added a patch for private message bug in phpBB 3.3.0. Later, I installed the Milk V2 style and extension. I had to remove the extension for the old style in the database, and disable Javascript to install it, otherwise there was an Ajax error. Later, had to copy ajax.js to the Milk_v2/template folder to resolve a HTTP 500 error. It turned out there were lots of other files needed, so copied all missing templates from the prosilver/template folder and also H2O folder from which it inherited. Basically, the Milk style was not engineered very well.
  • Some years back I helped a client move his static HTML pages into WordPress. A page in WordPress stopped working because PHP 7.2 was now the default PHP required by the web host. The members page returned a 404. Determined the reason for the latter was because page was in draft in WordPress. Published it. Discussed if the maintenance page should be reengineered as it was a custom PHP script that got content from a database table. He agreed to try Tablepress WordPress plugin to show this data so I managed to move the data in the table into a Tablepress table by exporting it to a CSV file and importing that into Tablepress. I changed Tablepress’s styling to fit the content on the page. For a while the table sorting and filtering feature doesn’t work. I discovered there was invalid HTML embedded in the table’s content. Fixing this solved the table’s sorting and filtering issue.
  • Client lost his old board site when his hosting lapsed. He wanted to recover the forum and get back as much content as possible on a new domain from some old backups he had. In this backup, I was able to find a phpBB 2.0.22 database from 2009. Apparently this was better than nothing, so I converted that to phpBB 3.3. To import the data, I had to remove rows from search tables using an editor due to bad data in them. This was no matter since it’s easy to recreate the search index. I provided the phpBB 3.3 files and database extract using Google Drive. He is trying to load them into his local XAMPP instance. Also spent 45 minutes in Zoom discussion about hosting and his various issues. Suggested some hosts to look into including Dreamhost. There is likely to be more work on this in July.

May 2020 work summary

May was a reasonably active month full of work, but most of it was for one client, a continuation of the work for this client for several months now. They are not in a particular hurry on their project and neither am I, but they do want it done right, so they’ll take the time to make sure it is done right. We are iterating on the styling in phpBB 3.3 for their current phpBB 3.0 forum with the goal of making it look as close to what they have now as possible. By mid month I felt like the styling was close to being done so stopped work, waiting for a review by the company’s designer. That took nearly two weeks so things were slower toward the last half of the month. I finally got the review last Friday and started doing some more work on it this weekend. Basically we’ll keep iterating through the styling until they are happy, then work on the next phase of the project, which will hopefully involve creating extensions to replicate what they are doing now through their custom modifications. The goal is to see if everything can be done through an extension. That’s similar to my approach with the styling: putting all changes into a custom style that inherits from prosilver, so most changes over time to prosilver will get applied automatically.

I’ve also been working slowly on a spam remover extension using Akismet. There is already an extension that will check new posts for spam and prohibit those, but nothing to clean up old spam. This is a continual issue for some of my clients but there is no way to find these and pull them out reliably. I’m hoping my extension will do this using the Akismet service. As I iterate through it though I keep changing features and trying different technical approaches. Since testing every post on a large board will be very time consuming and probably cause ugly PHP timeouts, I’m trying to figure out a way to do this more asynchronously. I’ll probably use an interface similar to creating a new search index with a screen that refreshes periodically to show progress. This is hard to do in phpBB.

Other work in May:

  • I performed a painful upgrade from phpBB 3.0.8 to 3.3.0. The board used the standard prosilver style, so I just had to add a logo. Sounds easy, right? But there were all sorts of issues with timeouts and things not working optimally. When the basic upgrade was done, I installed my digests extension and tested it. Installed also installed the Board 3 portal extension, which needed a patch to make it work on phpBB 3.3. I also set up reCAPTCHA V2 invisible spambot countermeasure and disabled contact form.
  • I converted an old phpBB 2.0.19 board to phpBB 3.3.0. There were not many posts on this board. The client chose the CleanSilver to replace the old subsilver style. I replaced the phpBB logo with old logo, and made small changes to CSS to show site title and description and to properly size the logo. I disabled the contact form, recreated the search index, and replaced the GD CAPTCHA with reCAPTCHA V2 invisible. I had to create my own account to login. cPanel access wouldn’t work for a while, and I got two different login screens.
  • Upgraded a board from phpBB 3.2.5 to 3.3.0. Eleven extensions were upgraded. Updated the Platinum style to a 3.3 compliant version. A number of extensions had to have their services.yml files edited to work on 3.3.
  • I dad a one hour requirements discussion on Skype with a client. If it goes forward, I will help move a board from a Delphi forum to phpBB through a lot of complex scripts. They can export to YAML to that has to be moved into a database and munged to be usable to phpBB. The basic idea is to import it to phpBB 2, which is relatively simple and if successful upgrade to phpBB 3.3 from there.
  • I upgraded a board from phpBB 3.2.8 to 3.3.0. I applied a private message bug patch. I reapplied their logo, which does not include old Site Lock code in the footer. I upgraded their Cleantalk extension to version 5.7.2.
  • Rehosted phpBB 3.1.6 forum from Bluehost. Lots of issues with Bluehost. Why? Because the Bluehost hosting environment is so tightly optimized that things you would expect, like to make a reliable archive in the file manager or to be able to archive the database easily in phpMyAdmin would time out instead. When I was able to piece together their database from various fragments, I moved it to a virtual machine on my computer to run database cleaner, then moved it to the Mac to upgrade it, then imported the database on Bluehost, with issues of timeout on phpMyAdmin loading the database too! It was all a big pain because Bluehost is a poor host in general, despite their slick advertising. This is probably because it is owned by Endurance International Group, which buys up hosts and squeezes every nickel out of them and their customers. I had to work through their legendary poor technical support to get FTP to work which involve chatting virtually with some guy in India. PHP 5.6 used, changed the config.php file to mysqli. Created an add on domain to do the work. Search index recreated as it was too big to move due to Bluehost’s phpMyAdmin timeout issues, so I had to recreate it manually.
  • Upgraded board from phpBB 3.2.2 to 3.3.0. Updated the Allan-Style SUBSILVER to the latest version and applied a patch to the style that was recommended to use it on phpBB 3.3. I also updated the board rules extension. All looks good.
  • Upgraded a board from phpBB 3.2.9 to 3.3.0. Updated my digests extension to 3.3.1. Upgraded PHP to 7.2 from 5.6. No issues. Client had concerns about emails not being received. There were some items in cPanel’s email deliverability report. Client may not have his phpBB notifications set correctly. Client did not have founder privileges to I granted them.
  • Troubleshooting. The client could not move or edit posts. The issue was that there is no user moderator role defined for him, which became a NEVER blocking permission. Made his role full moderator for all his forums and it seems to fix the problem.

April 2020 work summary

The number of clients I served in April dropped compared to earlier month. Last year April and May were slow too. However, I had one well paying commercial job that more than paid the bills, and should continue to do so for many months to come. It’s nice to have steady work in tough economic times. It will take a few more months to see if the economic crisis is affecting the overall customer requests I get.

This month I also discovered some new information on doing crons properly in phpBB not covered in the Wiki, but I can’t seem to change the information in the Wiki because logging in no longer works. This can affect my digests extension which can use a cron to send out digests. I’ve also started work on a spam remover extension, that will remove spam posts and private messages using the Akismet web service. Hopefully I’ll have something to show this month.

Work on my book continues at a slower pace. I’ve done two more read throughs of it, finding errors and corrections and am now on what I hope is the final read through. I’ve changed the book size to 7.5 inch by 9.25 inches, the standard size used for Dummies books. As there are lots of screenshots, it was important that they be legible. We’ll see how it goes. Hopefully I can release it this month. I still need a cover illustration.

  • I upgraded a board from phpBB 3.0.14 to 3.3.0. As part of this, I also upgraded PHP to 7.2.29. The board has 408,000 posts. Added extensions: Google Analytics, Precise Similar Topics and Topic Preview. There were issues installing extensions. For two extensions, I had to create a module category suggested by the error message first. These were installed as mods on phpBB 3.0, so that was likely the cause. The Google Analytics extension wouldn’t install using my Vivaldi browser, but would using Firefox. Go figure. I disabled the contact page, set up reCAPTCHA V2 Invisible spambot countermeasure, and created a home link. I had to make myself a founder using the database. Installed the Anime style and its extension. I investigated upgrading MariaDB to a newer version, but it requires command line instructions. A few days later I fixed an issue that brought down board by removing trailing comma from allowed_schemes_links configuration variable using MySQL and manually purging the cache. I turned this is a phpBB problem report, documented here, which looks like it will get fixed in phpBB 3.3.1.
  • Earlier, I attempted an upgrade to phpBB 3.3.0 that I had to roll back because the host (Network Solutions) didn’t have a PHP version you could select greater than 7.0! Unbelievable! Upgraded and reenabled the ShareOn extension and installed Thanks for the Posts extension.
  • I added new audio control to a board that features a streaming music player, since the old service seemed to go defunct. Some testing with the provided audio players, which were too big on the board’s header, resulted in a simple link to the music service instead, that opened in a new window.
  • Placed a new ad at top of page using Advertisement management extension. This included uploading a new ad image. I disabled the old ad.
  • I attempted to add Facebook tracking code to a custom style. I copied overall_header.html template from the AllanStyle-SUBSILVER template folder and made changes in the custom style. I tried purging the cache multiple times but the code never appeared. I even rebooted the web server. Eventually, I figured out the underlying reason was that the block of code I added was placed inside a S_ALLOW_CDN statement, before the <!– ENDIF –> statement, and since the template switch wasn’t getting executed the Facebook tracking code wasn’t getting added.
  • For my commercial client so generous to pay me my commercial rate, I spent much of the month working on this project. In March, I made an unsuccessful attempt to upgrade a test version of the database. The client made a new version of the old phpBB 3.0.11 database that was better sanitized, tried again and eventually succeeded. To convert the database, I need a PHP 5 environment, so it was hard to do on their server so I had to do it on my machine in a virtual machine, and that added time and hassle. I then began to try to retrofit the styling to make it look as close as possible to the existing styling. This meant hiding certain features and disabling others. The old styles used the Bootstrap framework and SCSS. The idea is get away from Bootstrap since styles are now responsive in phpBB, but it’s easier said than done, but otherwise keep the SCSS. Went through two iterations for client feedback, and there will likely be a couple more. In some cases I am using extensions to replace functionality, like the Board Announcements extension to see general text on the index. This is pretty challenging work and should keep me well employed probably through the end of the year. Likely in May I will be doing mostly styling work. I am using a custom style, inheriting from prosilver, to minimize the impact of future updates to phpBB. When the styling is done, there will be extensions to write to incorporate lots of additional functionality.

March 2020 work summary

This month was pretty busy with work, but also distractions. One primary distraction was trying to figure out why some people were not getting digests from my digests extension. A user was kind enough to give me access to a test board where I could troubleshoot the process. Over many days I discovered the issue was due to changing PHP “continue” statements to “continue 2” statements in the last release. I removed this code because it doesn’t seem to be necessary. Its purpose was to prevent duplicate digest from going out, which users had reported. But with a phpBB or system cron, it shouldn’t matter. The cron logic itself should not allow this to occur for these types of crons. Removing it seemed to fix the problem. I hope to get out a new release of digests in the next few days.

I’ve also been working on a book with the title Mastering phpBB Administration. It finished the editing process late this month. Now it needs cover art, possibly some reformatting for the printed page, then publication and marketing. I expect it will be mostly downloaded as an eBook. I’m hoping I can do this in April. I’ve invested several thousand dollars so far and I’d like to start seeing a return on investment. It’s my first attempt to write a book, and I can appreciate all the work it takes to do it right now.

  • Upgraded board from phpBB 3.2.6 to 3.3.0. There was a Javascript issue with an integrated streaming radio player, accessed through a control I placed near the header of the board. The client needs to bring a support issue with streaming vendor. It’s likely that the Javascript has to change. I had to copy the Javascript for the player back into the templates overall_header.html and overall_footer.html. I replaced the reCAPTCHA Checkmark keys with a set of reCAPTCHA Invisible keys, since phpBB 3.3 only supports the Invisible keys. I had to upgrade Tapatalk extension to latest Beta version because of an error after the phpBB upgrade that disabled all extensions unless I upgraded the extension. The web host control panel information provided did not work but I was able to login and upgrade using FTP. I was asked to recreate search index but 700mb of database space was in use and Siteground’s quota is 1GB. I suggested seeing if he could buy more space. Later in the month, I moved his domain and board from Siteground to GoDaddy where there is no 1GB database size limitation to the database. The board has close to 800,000 posts. The archive of files was 2.8GB. I then recreated the search index for the board. All went smoothly, which was a bit of a surprise given its size. I didn’t have permissions to move domain, so left that for the client to do. The audio player is still an issue. It appears the streaming service went out of business.
  • I upgraded a board from phpBB 3.1.9 to 3.3.0. Since I last worked on it in 2016, it was moved to a subdomain. I had to upgrade the version of PHP since at least version 7.1.3 was needed and version 5.6 was present. I updated the German traditional and German casual language packs. To address an umlaut issue, I changed the collation for phpbb_posts.post_text column to utf8_bin and created sample topic to verify it fixed the issue in new posts. I then fixed issues with umlauts in many tables using the MySQL replace function.
  • I placed an ad to appear at the top of pages using the already installed advertisement management extension. It took some trial and error to get the CSS right so that it centered properly. Later, I fixed an issue with various WordPress blogs on the site. The /wp-content/uploads folder permissions were 755, changed to 777. There is still an issue related to updating these blogs that I am still working on. Certain permissions are missing.
  • I upgraded two boards for the same client to phpBB 3.3.0 from 3.2.1. On first board, I updated the prosilver_se style and two German language packs, then made a custom style to ease future maintenance by encapsulating the unique changes in the custom style. I also disabled the contact form and installed a set of Invisible reCAPTCHA keys. On the second board, I updated the Latte style, reapplied minor changes to .sccs files in one template then recompiled the style. I also updated my digests extension from 3.2.6-RC13 to 3.3.0.
  • I upgraded a board from phpBB 3.0.14 to 3.3.0. To do it, I had to remove a number of dead extensions and modules from the database. I moved over digests subscribers from the digests mod. I tested a new cron job used to kick off digests, but the manual mailer did not seem to work properly. I installed the ACP Add User extension. I disabled the contact page. I created a reCaptcha V2 Invisible spambot countermeasure. There were AJAX issues installing extensions related to a CORS error. After doing it enough time, the extensions seemed to install correctly. Later, the client had a script that used data in the user_interests column which was not copied over. I populated it from my backup, but the script should reference the phpbb_profile_fields_data table instead, where the data now resides. Client was concerned about &amp; that appeared as text in digests. I removed a line of code from my digests extension that added this entity as it no longer triggers a validation error.
  • I upgraded a board from phpBB 3.2.8 to 3.3.0. I had to disable modSecurity in the .htaccess file to get into board after upgrade. Updated the advertisement management, Cleantalk, Board 3 portal and Tapatalk extensions. Tapatalk would not work on 3.3 without upgrading to Beta version 2.1.7. Changed the board’s .htaccess to redirect to portal by default. I reapplied the logo. i changed the spambot countermeasure to reCaptcha V2 Invisible.
  • Client had issues with a custom style created by another developer when his board was upgraded to phpBB 3.3.0. The simple solution is to copy prosilver’s templates to the custom style and purge the cache. Instead, I made a proper custom style that inherited from prosilver. Only the CSS changes needed to be applied to prosilver are now in the custom style, making it easier to upgrade the board in the future.
  • I upgraded a board from phpBB 3.2.7 to 3.3.0. Client had issues doing it himself using the automatic upgrade method, which should not be used for phpBB upgrades, but only updates.I had to change PHP to version 7.1. A redirection issue on main .htaccess file for site meant I had to rename it temporarily to do the install. Changed reCAPTCHA spambot countermeasure to use the invisible CAPTCHA. Had to update Cleantalk extension to 5.7.2 to get it to install.
  • I do ad hoc work for an existing client as user issues come up. In March:
    • I changed an email for a user to her gmail address. Later, I heard there was another problem with her access, but when contacted she said she was getting email notifications.
    • I informed another user of her username and provided lost password instructions.
    • I forced reactivation for another user who could not login. It didn’t seem to work, so worked over the phone to get him online.
    • Changed email address for another user.
    • Provided instructions for a user to get a new password.
    • Advised the board’s administrators on a DNS issue sent by their hosting provider.
    • Provided lost password assistance to another user.
    • Changed the email address for a user and provided resetting login procedures.
    • A user wasn’t getting email notifications. Asked her to look in her spam folder and see if they are there. Eventually figured out her email address was incorrect for receiving notifications and changed it.
    • I created a new password for a user so he could login.
  • This month in a long term project I am doing for a client, I was trying to do a test upgrade of their phpBB 3.0.5 database to phpBB 3.3. My first job is to update the style to match their current style. Both attempts failed:
    • In the first attempt, I realized that I needed PHP 5.6 to install the support toolkit to remove bad data from the database. I tried to install it myself on their machine but not being a system administrator, it didn’t seem to work.
    • I next downloaded their database and tried to convert it on my laptop on which I had installed PHP 5.6. The database eventually imported (four hours later) but the laptop is underpowered. phpBB’s support toolkit flaked out when I attempted to use it. An attempt to run the upgrade program didn’t work very well. The client made lots of changes to the database that had to be cleaned up, but it would still not upgrade properly. PHP 5.6 was not the best choice. I really needed PHP 5.2, since trying to upgrade I had to work around issues making phpBB 3.0 work with PHP 5.6. It’s getting harder to set up test environments that use older versions of PHP.
    • I then imported the database into a virtual machine on my Mac, and a reference version of phpBB 3.0.5 as well so I could use its database to compare with the client’s. I realized I needed to understand all the changes made to the database, because I may need to leave data in the database because it might be needed when moved to phpBB 3.3. I have a call with the client today to look over these changes and factor them into my next test upgrade attempt.

February 2020 work summary

This was a fairly ordinary month, but I have started two long-term projects, details of which are below. Both should keep the business comfortably afloat for much of the year.

Work on my book Mastering phpBB Administration continues. It is still with an editor but should complete editing within a few weeks. It still needs cover art, reformatting, and setting up a publish on demand mechanism. I’ll probably offer it as both an eBook and a printed book. Existing clients will be entitled to a discount. Figuring out the pricing will be challenging. At over 300 pages, it’s a lot of information. Publish on demand allows both color and black and white copies to be requested, but color copies are much more expensive to produce, so I may offer both at different price points. Price will probably not be less than $19.95 a copy. Stay tuned for more details.

  • Began an exploration on whether a Delphi forum could be converted to phpBB. The client provided a set of .yaml files with the board’s posts, which also includes some images that were in the posts. My basic approach is to see if I can get the data into a format that can be converted into the old phpBB 2 database format successfully. Is so, then it could be converted to phpBB 3.3, the ultimate goal. The first step was to try to find a good YAML parser. I tried a number of approaches but eventually I used Symfony’s yaml parser in my script and that seemed to work reliably. The .yaml files were not perfect YAML, so I had to do a bit of editing to get files to parse correctly. I parsed the first 500 posts and placed the result in a HTML table with a script, then did the same thing with a .yaml file with the user information. These were sent to the client. The client will discuss with his group whether he wants to fund further research, but it looks likely because it looks like there is enough data in these files to populate a phpBB 2 database. He will get back to me in a couple of weeks.
  • A client was running out of quota and asked me to figure out how to solve the issue. 4.9GB out of 5GB was in use. Problem was WordPress backups in a backup folder. Suggested removing old backups. Once identified, client took care of issue.
  • After large upgrade to phpBB by a client, the modules for my digests extension got lost. On Skype, I walked him through the process of creating both the ACP and UCP digest modules manually.
  • Upgraded and rehosted board from phpBB 3.0.5 to phpBB 3.3.0. The new hosting is on Siteground. The old hosting was on a Windows server. Fortunately, MySQL was used on the Windows hosting, so there was no database to be converted. There were some technical issues on both hosts with downloading databases and making archives. Siteground’s new site tools turned out to be confusing as cPanel was not an option. I could not grant permissions to the database user I created to database I created. It took two tech support calls but eventually it all worked. I created a custom style to show logo, otherwise default prosilver style was used. I disabled the contact page and created reCaptcha V2 Invisible spambot countermeasure.
  • I provide ongoing support for a client’s business board. I investigated an emailing issue where emails were getting flagged as likely spam when sent to wvu.edu. The client’s domain is not on an email blacklist. This was something on wvu.edu’s end and the easiest way to deal with the problem was to have change the board user’s email address. I provided instructions on how to change email address used by the board. At the end of the month I updated two packages in Plesk and updated Plesk to version 18.0.24. Also, changed one user’s email address. She was not getting email notifications because the address had changed and she didn’t know how to change it in phpBB.
  • I restored a lost board. Originally it looked like a backup of phpBB 3.0.11 was available, but it wouldn’t upgrade since some tables were in a phpBB 2 format and some not. The client found an older phpBB 2 database backup and I was able to convert that offline to phpBB 3.3. In the first conversion, the attachments were missing so I had to do it over again. I also changed the board to use to https. A few minor errors were encountered. I had to add an auto increment to the phpbb_logs table’s primary key. I set up reCAPTCHA and disabled contact page.
  • Updated forum from phpBB 3.2.2 to 3.3.0. I had to comment out Options -MultiViews in .htaccess file to avoid HTTP error. Also had to remove install_config.php file in the store folder to get the upgrade to start. Otherwise there were no issues. User had warnings that were due to PHP 7.3 being enabled on his host, which is not supported by phpBB 3.2.
  • Began work for a new long-term client who has a large job to upgrade his phpBB 3.0.11 board to phpBB 3.3. This will take many months as there are many customizations embedded in the source code itself that need to be turned into extensions. This month I mostly looked through their changes to phpBB 3.0.11 and advised on the best approach. The first step will be to try to change the style to work with phpBB 3.3. The style uses SCCS and needs to link in a lot of CSS and HTML used for the master website. I may develop some of the extensions myself of work with other extension authors to have some develop them. Some look very complex, such as a proprietary OAuth integration. But it’s commercial work and pays very well, so it should be both challenging and financially rewarding. More on this in coming months.

January 2020 work summary

I did plenty of work in January, just not so much for clients. That business was slow, but picked up toward the end of the month, so much so that at the moment it looks like my plate in February should be full.

Why was I busy? I was writing and editing a book on phpBB administration. The introduction of phpBB 3.3 Proteus on January 6th added work, as I needed to update the book to talk about phpBB 3.3. As I noted, its feature set makes it more of a micro release than a minor release, so that part was good. But by changing the phpBB logo and a few other things, I had to recapture fifty or more screenshots, which was tedious and time consuming. Then of course the text had to be revised in places which also meant reading it through twice more. Right now it stands at 305 pages on 8.5 x 11 inch paper.

I reached a milestone of sorts when I turned it over for editing on Thursday. I should mention that I am using Maria Williams, so if you need an editor, hit her up! It will take her a couple of weeks for her to do her magic, then I have to figure out how to publish and market the book. I need cover art. Publishing will likely be on demand, and I expect most people will want electronic copies. I also need to figure out how to price it. It’s a sizable capital investment of time and money, so it needs to be profitable. I expect to offer some sort of discount for the first ninety days and for my existing clients. The last book published on the topic was written in 2006, so it described phpBB 2, so the market is out there. Also, that was a user guide, not an administrator’s guide.

Otherwise:

  • Added two users and forced a reactivation of two other users for a commercial client, who outsources this work to me.
  • Updated a board from phpBB 3.2.7 to 3.2.9. Also, I created a new forum with a forum logo on the board. I resized and placed a forum image logo with the forum, and put it in the requested place on the index. I gave the forum permissions similar to another forum. I investigated issues in the error log. Most of these errors seems to be due to debug being turned on.
  • More analysis for a client I did work for in December. I generated a list of users and email addresses on his archived forum that had contributed at least 1 post but whose usernames were not on the current forum. He will use the list to invite them to come to the new board. I installed a release candidate extension to creating lots of user accounts in bulk, but it didn’t work as advertised. I did some styling changes. I upped base font 1 pixel to 12pixels. Client wanted to know if my digests extension can show text only. Yes, and I sent him a sample. He wants digests generated by my extension to look similar to someone else’s digest. I deleted a user defined moderator group. I provided guidance on using the ACP Add User extension so lots of old users could be added. He would like to delegate the work to someone else, so discussed how this could be done with ACP permissions so their access was tightly limited in the ACP.
  • Client’s board wouldn’t come up on Bluehost hosting. After some analysis, I determined he had a phpBB 2.0.19 board and that only PHP 7 was available. He did have attachments. I converted the board to phpBB 3.3.0. Issues logging on were solved by removing the domain’s cookies, clearing browser cache and resetting the password, which was set back. I moved the old board into forum_old folder. I created the search index, disabled contact page, added Invisible reCAPTCHA. The resulting board is very basic. It may need some customization, styles and extensions installed. Later there was additional work. The client needed some gallery software. The phpBB Gallery extension is not being maintained and is not reliable, so looked for a solution outside of phpBB. I tried one solution, then tried another. Piwigo seemed to work after I installed a Community plugin/extension. This allowed users to upload their own pictures.
  • I updated a board from phpBB 3.2.0 to 3.2.9 on generic prosilver style. I updated my digests extension to version 3.2.17 and the Cleantalk extension to version 5.7.2.
  • I upgrade board from phpBB 3.2.7 to 3.3.0. I updated the prosilver SE style to 3.2.9 and put the old logo back. I changed the spambot countermeasure to reCaptcha V2 invisible, since spam registrations were a complaint.
  • Client wanted active topics in red on the navigation bar. First tried installing the Advanced Active Topics extension (release candidate) but couldn’t get it to show. I then installed the Recent Topics extension as a trial, but client didn’t like it, so I created a special link on the navigation bar, highlighted in red, moving it from the Quick Links menu.
  • I upgraded a board from phpBB 3.2.8 to 3.3.0. I replaced the logo.I checked all the extensions and they seemed to work on phpBB 3.3 with no issues, but there was no test case for change post time extension to verify it worked properly.
  • More work for a client on spam. Used Prune Users feature to remove users and their posts who registered after Feb 10, 2018 when spam seemed to start in earnest. Reenabled board and suggested ways to mitigate the problem in the future with moderation, digests, etc. This is discussed in more detail in this blog post.
  • This work is still ongoing. On Hostgator hosting, I created space for five domains and set up DNS for these domains. I set up five boards on five subdomains of these domains. All have a basic configuration, registration disabled, reCaptcha enabled, prosilver_se style applied, contact form disabled, and forwarder set up to redirect to client’s email address. Set up https redirection by installed WordPress SSL plugin on the main blog. Next, I cloned four WordPress instances based on work on one domain, whose theme and plugins client wanted to replicate. I did this by copying the files and database, changing the wp-config.php file to point to the new database, and tweaking the site information in the wp_options table. This saved a lot of time and work for the client.