May 2018 work summary

Things were a bit slow in May, at least for the first two weeks. Curiously all my work was for existing clients. The theme of my work in May: getting rid of spam and preventing it.

  • Client #1
    • May 4. I helped fix an outgoing email issue on GoDaddy hosting. phpBB notifications were being blocked by GoDaddy from going out, something that you would never know if you don’t call their technical support line and press them. GoDaddy detected malware on the domain but after running a scan could not find any. They required that the customer certify that there was no malware on the site before unblocking outgoing email. In response I created an email forwarder for the domain and made this the board contact email. Not having an email address for outgoing mail associated with the domain is often the root of these problems. Customer called GoDaddy, they reenabled outgoing email and notifications started going out again.
    • May 12. Issues with spam posts on the forum despite using the reCaptcha V2 spambot countermeasure setting in phpBB. Customer was paying extra for a GoDaddy Securi service that automatically detected and removed suspected malware, but which removed some valid software introducing issues. Securi zeroed out some software files that kept me from logging in. After I eventually figured out the problem, I replaced the two files and I was able to log in. Earlier I had removed copies of mods in the store folder (used when the board was using phpBB 3.0 software) that were flagged by GoDaddy as malware. There were recent spam posts and it’s unknown how they happened. It’s possible the spammer solved the reCaptcha manually. I deleted about 150 inactive and likely spam registrations via SQL. I provided user with a list of posters since 4/1/18 so he could make a judgment as to whether these were spam registrations. I provided instructions on removing these users and their posts. I reinstalled Tapatalk since it was not working. I had to do it through File Manager since FTP wasn’t working. mChat extension wasn’t working either. I tried to reinstall that too but it didn’t work. I needed to reupload all phpBB software files since components were probably removed by Securi. Noticed navigation icons were removed too. Talking with GoDaddy they disabled the firewall and got me FTP access again. I re-uploaded folders where Securi had changed things. This seemed to fix remaining issues. After purging the cache, mChat started working and icons reappeared. Uploaded Cleantalk but not enabled. Providing instructions for configuring it if client wants to use it. Had GoDaddy disable firewall. Customer may want to stop paying for it.
  • Client #2
    • May 12. Since my last work for client they moved their forum to a dedicated server. Could not get in with FTP credentials. Used file manager instead. Installed release candidate of Thanks for the Posts extension. It will work with Tapatalk if the mobiquo directory is replaced. Did that and tested and it seemed okay. Installed a beta Modern Quote extension. Hid the portal page link because a sidebar would not show on right due to styling issues. Changed server settings, cookie settings and .htaccess file to accommodate HTTPS. There were some issues with getting cookie settings to work correctly. I also had to change some URLs in the posts table to https and fix some ads that were showing http in the URLs. To accommodate a Discord chat widget client wanted on the sidebar, I created a .css file it uses and linked it in in overall_header.html. I placed it at the bottom of the right sidebar in a custom widget in the Board3 portal, but there were connection issues since the service was not authorized that the client fixed. I fixed a phpBB Gallery integration issue. In the config table, the row containing phpbb_gallery_num_comments was blank when it needed an integer, trigger a SQL error. Set it to 1 since there was only 1 comment in the gallery so far. Fixed edit mchat post issue so an admin could edit the chat post due to white text on white background. Applied mChat patch to force the window to scroll to bottom in mChat. Fixed background and text color of Discord widget with an inline style to the widget since the board portal stylesheet was overruling it. Clock widget bottom part did not refresh on Chrome, but does on Firefox. Not sure it can be fixed (browser quirk) so client removed it. Checked background color issue for multi-quote. It’s black for me. Does not appear to be permissions based.
    • May 13 – Changed the background color for Gallery display, was white, commented out in CSS. User provided a watermark and I added that. Changed cookie name, forces a new login.
    • May 14 – Sorted through issue with Gallery watermarks (should be applied to subsequent images) and virtual URLs it creates. Disabled Image upload extension.
    • May 16 – Mostly answered new questions, changing some settings such as a PHP maximum upload size. Installed Search User Topics extension.
    • May 28 – Client asked me to update the version of Tapatalk to 2.1.10. However, it didn’t integrate well with version of mobiquo supporting the Thanks capability. To get things stable, we eventually reverted back to Tapatalk version 2.0.8.
  • Client #3 
    • May 16 – Time consuming phpBB upgrade on dreamhost.com virtual private server from phpBB 3.0.14 to 3.2.2. Over 1.2M posts, took 3 days! Running a database update quickly failed. Tried the command line interface instead. Even then it was excruciatingly slow. Broken SSH connection overnight meant a restart which bogged down. A chat with tech support finally revealed the upgrade slowness issue: insufficient RAM for the database. Bumped it to about 600MB and the conversion finally completed. Everything was straightforward thereafter. Used the black style which I changed to make red type rather than blue to match the look under phpBB 3.0. I replaced the logo. I added the latest Tapatalk extension and enabled it. I added Cleantalk extension and gave instructions on how to enable it with a key from the website. Turned on its spam firewall feature. I created a robots.txt file to deny all robots. I enabled reCaptcha V2 spambot countermeasure. Changed group color for administrators so they could be seen easily in styles.
    • May 17 – Additional work. MySQL too many connection errors kept happening. Spent time with tech support on this issue. MySQL settings cannot be changed for the VPS type of account. Some users were reporting bogus login errors. Disabled Cleantalk and think that will solve the issue until Cleantalk is properly set up. Added prosilver_se style for visually impaired users who can’t see dark style well. Installed Quick Style extension so styles could be easily changes. Changed blue text colors to white but cannot change color of icons since they are images — would need new set of icons colored correctly. Client wants to jump to the first unread post of a topic when clicking on topic icon. Found a hack for it but advised against it.
    • May 18. Investigated some new issues. The search index unavailable message when posting, but post was saved. Changed search index to phpBB Native. Installed Naruto shipuuden style and tweaked it to place logo and make it look right.
    • May 19 – Answered some questions. Installed Thanks for the Posts (Release Candidate) extension. I did more tweaking of newest style to make it look more uniform.
    • May 20 – Answered a lot of questions. Think the too many MySQL connections issue is either related to Tapatalk’s mobiquo directory or needing to change the storage engine for tables to InnoDB. Updated Cleantalk extension to latest minor version. Changed PHP from 5.6 to 7.0. Uploaded version of mobiquo for Thanks for the Post extension.
    • May 21 – Installed ShareOn extension. I found a way to get to MySQL prompt for potential change of tables storage engines to InnoDB. Changed a lot of forum permissions. Explicitly gave Thanks for the posts permissions where necessary. Verified it worked in Tapatalk with the replacement mobiquo folder they provided.
    • May 22 – I changed all table storage engines from MyISAM to InnoDB. The search wordmatch table took about six hours to convert. This resolved the MySQL too many connections problem, probably because tables were being locked and unlocked more efficiently and at the row level instead of the whole table level. I also installed the Media Embed plugin. I wrote a little blurb of instructions for client’s users and provided a screenshot for using the Media Embed plugin.
    • May 24 – I installed Advanced BBCode Box 3 extension to make it easier to embed videos.
  • Client #4. To address a persistent spam problem, I installed the latest version of Cleantalk and changed spambot countermeasure to reCaptcha V2.
  • Client #5. To address another spam problem, I installed Cleantalk extension on two domains.
  • Client #6. Client had a major spam attack with thousands of spam posts and spam registration. Configured reCaptcha V2 on phpBB 3.2.1. Then installed Cleantalk extension. However, there were issues while removing spam with Cleantalk, first hosting timeouts, then duplicate rows in phpbb_notification_types table. Emptying this table eventually worked. Removing spam was a combination of some SQL on the backend plus Cleantalk extension.

MediaTemple grid service no longer recommended

It’s sad for me to say this, but I can no longer recommend MediaTemple’s Grid service as a hosting option. For the last 18 months or so I’ve been using this service and have gotten increasing dissatisfied and exasperated by it. Today I started the process of moving my domains off of it to Siteground.com even though I have four and a half months left on that hosting contract. This domain should now be coming to you from siteground.com servers and hopefully in a reliable and maybe spiffy fashion.

MediaTemple.net was known as one of the premier providers of business-class hosting. It was acquired by GoDaddy with the promise that it would be separately managed. It appears from my experience with their Grid service that they broke that promise with their customers. Sometimes I have to wait a minute or more to retrieve pages from my own site. It sure looks like they are overloading their servers and/or managing them very badly.

I used UpTime Robot to test whether my domains are up. Pretty much every day I will get one or more emails telling me it is inaccessible. So it was likely costing me money, motivating me to move to Siteground.com instead.

I do have clients using MediaTemple’s virtual servers and they have no complaints about that service so far. Definitely avoid their Grid service now and if you have an option you might choose some other host for your virtual or private server needs. I don’t have experience with Siteground’s, but it’s likely fine. A company like Rackspace.com is likely doing it right.

Another annoyance was revealed simply in moving my site. An old phpBB forum I have with about 50,000 posts could not be downloaded. I had to break it down into multiple downloads, including the posts table into two separate downloads. The Grid service simply cut me off when I hit some sort of resource limitation. The whole database is only 80MB or so. Shame!

April 2018 work summary

In contrast to March, April started out busy and ended up slower. This is in part because much of my March work continued into April. Anyhow, here’s what I worked on for clients in April. As always, I do not identify my clients by name or domain unless they specifically allow it:

  • I spent a lot of time continuing work for a client I started working for in March. Their WordPress site integrates phpBB into a separate area.
    • First I completed the move of the forum from phpBB 3.0 to 3.2. The hard part here was making changes to the phpBB style to integrate the overlaying WordPress theme in the headers and footers, as there were many CSS class and HTML ID conflicts.
    • As part of the upgrade, the client took the plunge and moved the site from standalone web pages to WordPress. A lot of scripts written by someone else to monetize particular types of posts broke because of changes to phpBB’s API. These old programs were “tightly coupled”, to use a software engineering term. Basically as these broke I fixed them by updating the phpBB code inside these programs. Most of this work involved changing $_SERVER, $_GET and $_POST statements to use phpBB’s request object instead. 
    • With the new WordPress framework, some of the old content appearing in sidebars needed to move into WordPress widgets in the sidebars of the theme. We’re taking this piecemeal, learning as we go along. A better “loosely coupled” way to get phpBB content in WordPress is to use phpBB feeds. phpBB comes with a built in Atom feed that needs to only be enabled. But it doesn’t do everything to retrieve relevant posts. WordPress has a feed widget and there are also a number of other widgets available as plugins that do a fancier job of serving feed content from Atom or RSS feeds inside of WordPress. In the case of my client, the sidebar needed to show recently posted topics in the forum. I eventually decided I needed something beyond phpBB’s Atom feed, so I chose my own Smartfeed extension instead. However, it didn’t return quite what the client wanted as he only wanted “fresh” topics from a certain forum, and only the latest post from these topics. I was able to tweak my Smartfeed extension to do this, and this inspired me to work on a new release of Smartfeed to add these features. Once deployed, I will update the version of Smartfeed I installed for this client to use it instead.
    • I fixed an issue with moderator permissions with the forum by adding these moderators to a list of global moderators.
    • Installed a SiteMap extension so posts were better available to search engines.
  • Upgraded a forum from phpBB 3.2.0 to phpBB 3.2.2. Replaced the logo in the original dimensions. Upgraded Cleantalk to latest version on GitHub. Enabled the spam firewall feature of Cleantalk. Installed announcement on index extension for 3.2 (not official). Fixed issue with wrong reCaptcha. Checked global moderator privileges and they were okay. Administrator moderator role was changed to Full Moderator.
  • Upgraded Tapatalk extension to version 2.0.8. This also meant removing the mobique directory and reuploading that.
  • Changed the people who have moderator privileges for a client.
  • This work was a bit different. I created two MySQL triggers so if a user is added to a particular group by an administrator they are subscribed to receive notifications if new topics are created in a forum. I also created a trigger to remove the forum notification if they are removed from the group.
  • A client needed URLs embedded in the post text changed as the image hosting service has moved it to a new domains (postimg.cc and postimages.cc). It’s fairly simple to do once I finally figured out about the MySQL replace function. I also installed the Member Profile Views extension (1.0.3) for the client. Later, I updated the Cleantalk extension to the latest version. While I was in there removed 3 dead extensions from the extensions table that were showing as ugly error messages on the Install extension screen. I also help create a new BBCode to serve embedded videos from a site and updated the latest version of phpBB Gallery from the GitHub master: version 3.2.2.
  • Upgraded a forum from phpBB 3.0.11 to phpBB 3.2.2. Installed Allan Style Subsilver. Placed a new logo. Installed Advertisement Management extension and placed two old ads, changing the image URL a bit. The forum has 200,000 posts. One minor issue during upgrade: I had to make active the prosilver style for it to complete.
  • Completed a messy upgrade from phpBB 3.0.12 to 3.2.2 on GoDaddy hosting. The file folder was missing and was apparently dropped a month or so back when user went over quota, and was not recoverable. This meant lots of images embedded in posts would not show. The user has to live with this. I had to move the database to my machine to convert it, due to severe resource limitations on GoDaddy. The phpBB native search index creation did not work, so I created a MySQL Full Index one instead. mChat was previously installed as a mod. I had to manually remove all traces of it to get the mChat extension to install. Many tables were missing primary keys, indexes and automatic increment settings. I eventually figured out this was an issue with bigdump.php, a staggered MySQL importer I used, as it nears the end of a database extract file sometimes and hangs. These are stored at the bottom of the file and it never said it completed. It took an hour or so to create the missing indexes and primary keys. I did not move over tables for the search index, topic tracking information, the session information or the logs due to size issues. I could not find a way to remove spam posts easily but did remove old inactive users. Installed reCaptcha V2. Installed Tapatalk, Advertisement Management (to place ads at top and bottom), ShareOn, mChat (after cleanup) and OneAll Social Login extensions. Adjusted forum permissions so newly registered users could post without permission and gave full access permissions to all forums for all major groups.
  • As with the first client, I changed some URLs in the post text to use postimg.cc and postimages.cc. This one though was a bit different because periods were stored as HTML entities in the database. I also spent an hour trying to make the upload attachments feature work since he did not want to use external services anymore for posting images and he had plenty of space. There were lots of permissions issues to puzzle through. I had to change forum access privileges for registered users to full access. I also had to set the upload permission for the registered users to allow it and change the quota for uploaded images from just 1MB to 10GB.

March 2018 work summary

What a busy month during March! The first week was a bit slow but pretty much the rest of March I was busy serving clients, many of them commercial clients, which likely made it a record month for work billed, not all of which has been paid yet. A summary of the work is below. As always, who I did the work for has been kept anonymous.

  • Upgraded a forum from phpBB 3.0.14 to 3.2.2. After the upgrade, I could not show the index until I increased PHP’s max_execution_time by editing the forum’s .htaccess file. Installing my digests extension did not work, perhaps due to issues with bringing over data from the digests mod that was installed. Importing digests from phpBB 3.0 has been pretty extensively tested, so I’m not sure what the issue was. I ended up going into the database and correcting various tables and effectively installing and enabling it that way. I recommended installing reCaptcha as a spambot countermeasure.
  • A client was having trouble with WordPress shopping cart tied to PayPal. An email notification by the shopping cart was getting forwarded to a bunch of email addresses. Apparently the email address was set to forward all email it got to a number of other address. Changing the forwarder fixed the problem.
  • Updated a forum from phpBB 3.2.1 to 3.2.2. Uploaded a freshened prosilver_se style and Danish and German casual honorifics language packs. Updated Tapatalk plugin to version 2.0.7. Reapplying style changes required changes to both prosilver and prosilver_se. Later, I fixed an issue of an inability to attach pictures to posts by simply purging cache. Later, I configured the reCaptcha spambot countermeasure due to bounced emails being received. I suggested changing the user registration settings to user activation by email.
  • First I did some troubleshooting. A phpBB 3.0.8 forum could not display certain pages because PHP Notices kept coming up. Trying to edit these out with error_reporting statements did not work for some reason. There were some odd issues with the host too, such as it would not report which version of PHP was installed. Client eventually authorized an upgrade. The FTP credentials provided did not work, so I ended up creating two FTP accounts until one of them worked. This allowed me to upload a new version of phpBB. The upgrade succeeded after fixing an issue in the database of no active style. The preferred AcidTech style did not work with the large rotating images the client wanted places. So I tried a couple of other styles. We settled on a black/green ne-blackgreen style. I had to integrate client’s in-house ad rotatator program so a number of images would rotate in the header. I made some style changes, mostly to remove gradient images that were messing up the placement of the navigation bar. I added the Advertisement Management extension. I placed a PayPal donate button at the top. Integrating a Google Adsense ad did not work. I used the old code but think it’s too old as it was used inside a table. I did some extra work to put two other ads next to each other with different sizes. Changed max-width of the content area from 900 pixels to 90%. Later, I installed Tapatalk extension version 2.0.7.
  • Client moved the database and files for his forum to a new host, just needed me to finish up the work. I could not connect to the forum after changing the config.php file to point to the new database. This was because PHP 7 was being used and phpBB 3.1 uses mysql_connect, which does not exist in PHP 7. No matter. Uploaded phpBB 3.2.2 and the database update worked fine. I replaced the logo. I removed dead ACP modules. Later, I discovered I apparently I upgraded the wrong database (an older version of the forum), so changed config.php to point to the correct database and upgraded that with no issues.
  • Troubleshooting on a phpBB 3.0 forum. Database exceeded it’s quota of 1GB. The issue was a tapatalk_push_data table, which had 779MB of data in it. I truncated the table. I recommended updating the mod as it seems to be a bug or moving to phpBB 3.2 or rehosting.
  • Clicking on some thumbnails did not bring up the larger image. It appears there were some lost images due to a RAID failure in 2016. Some files in the files folder showed 0 byte length. Some files did not exist. I could not find a backup of these images. So I wrote a script to sort of fix the issue by making the thumbnails the actual photo: changing the filesize attribute in the attachment table to the thumbnail’s size and making the thumbnail column 0. In 88 cases the thumbnail files did not exist either. Provided client with a screen capture of all these abnormalities so he can decide if he wants to delete or edit these posts.
  • In part one of a two part job, I upgraded forum from phpBB 3.0.4 to 3.2.2. Since the forum was integrated into a larger commercial site, I did an extensive analysis of what would be lost with an upgrade. There were many code insertions into phpBB to do some things unique to the business. With the new architecture, this should not be done, but there were no extensions to do the previous functionality. Much of it revolved around a “Get a quote” service. The best i could suggest was to allow these installers to have standard signature lines that would embed this information and link to their referral program. Once we agreed on all the workarounds, I started off with upgrading the less trafficked site as the first test. I applied the AllanSubsilver style. I made some CSS and template changes to accommodate site heading and Google site search, changed some colors, changed some container widths, and address some font issues. I installed and configured the Google Analytics extension. I installed the Advertisement Management extension but did not add any ads at this time. I demonstrated the Get a Quote feature in a signature line. I removed contact information from view topic screen by hiding it. I did various other tweaks. I added the announcement on index extension. I configured reCaptcha and removed thousands of bogus registration and spam posts that quickly accumulated when the forum went live. I suggested they might want to use Cleantalk extension and service. Later I addressed a reCaptcha issue of the CAPTCHA image not showing upping PHP from 5.4 to 5.6. It turned out though that there was some other PHP software on the site that could not use PHP 5.4. PHP was reverted to 5.4. I found a setting in Plesk to set allow_url_fopen to on and the issue went away. More than a week later, I did part two (second domain). A lot of the work was the same as the first part, so it went somewhat faster since I knew what the issues were. Header was different with dynamic content scripts I could not integrate, so I copied/pasted the resulting code. Installed same extensions. Configured reCaptcha. Hid logo block entirely since that was the way it was on the old forum. Did not put fuzzy image along borders of main content area to match what was there before. Changed PHP to allow it to open external links. Changed script for main site to not retrieve a block of recent topics since the database schema had major differences. Likely more tweaks to come in April to close off this project.
  • I upgraded a forum from phpBB 3.1.10 to 3.2.2. Client has a ton of extensions. He decided to upgrade those himself and just have me do the upgrade. The style for Platinum for phpBB 3.2 was not official and I found issues when I installed it. For example, navbar.html did not have logoff and other links! So copied and pasted the version from prosilver and used that and it worked with the old style.
  • I finished an update/upgrade the client started but failed. I could not login to forum with credentials given. I could not use FTP with credentials given either, but was able to create some in cPanel. I tried a few obvious things to get the upgrade/update to finish that did not work. I ended up copying up the entire 3.2.2 source code and it was able to finish the upgrade/update. I placed the logo image using the File Manager editor in cPanel.
  • New install of phpBB on a test server that would eventually be migrated to a production domain. Client wanted to imitate the forum at another website and wanted it put together on the test domain. Some wasted time because I thought he wanted me to imitate his existing website. Most of the work was trying to make the styling acceptable. Client selected the AllenSUBSILVER style. I installed the Advertisement Management extension and placed a sample ad. I installed Pages extension because client wants a bunch of static pages like Contact, Ads, etc. They have placeholder text. Styling was challenging because I decided to use the navigation bar for his other pages and I had to make it red, but doing this made a lot of text and icons impossible to see. So I had to change CSS for submenus so the desired look was accomplished. 90 minutes of tutoring online followed some days later.
  • After an upgrade I did in February there were dead extensions showing on the Customise tab, extensions that worked in phpBB 3.1 but were not available for 3.2. I fixed the by deleting these rows in the phpbb_ext table using phpMyAdmin. Client asked if there were ways to improve performance. I suggested upgrading PHP to 7.2.
  • More work for a client I helped in February. He had weird database permissions issues which caused certain extensions not to install. He moved the forum to a different server, and that seems to have solved previous database permissions issues. Basically the people on this site like lots of features, and wanted me to install a boatload of extensions, many of which were problematic because they were in development, beta, etc. After being assured that I would be paid for extensions I installed that might not work on phpBB 3.2, I ended up installing a total of twelve extensions, some of which as I expected had issues and were subsequently disabled.
  • Upgraded a forum from phpBB 3.0.10 to 3.3.2. Installed Revolution style but tweaked it quite a bit. Background color changed from black to white. Increased minimum widths for content area. Reduced vertical space for logo and content area as it was too generous. Styled the search box to make it look similar to other content areas. Installed reCaptcha V2 as spambot countermeasure.

February 2018 work summary

The nature of my business seems to be that it the work is few and far between or it all comes in at once. The first half of the month was mostly quiet. The second half was mostly busy … too busy. Anyhow, here’s a summary of my work in February. As always, I am not identifying my clients:

  • A lot of work here for one client. Basically, the client wanted a whole lot of changes to his forum. The endeavor was sort of successful.
    • I installed a development radio extension. This puts a player on the screen that plays music from a playlist for all members on the forum. I had to tweak the composer.json file to get it to install. To get the radio player images to appear on the portal, I had to hack one of the programs for the extension to use an image path that starts with / or they would not appear on the portal.
    • I tried to install Tapatalk a number of times but to no avail. I kept getting an installer error creating the Tapatalk tables. Upgrading phpBB didn’t work. So I tried installing an earlier version of Tapatalk. That didn’t install either. Since the issue was with the Tapatalk table not being created, I installed Tapatalk on a test environment, got the needed SQL there using phpMyAdmin and recreated it manually on the client’s site in phpMyAdmin. I changed the phpbb_ext table to make it active and cleared the cache. I could then see it in the Extensions tab and could enter the key.
    • Created an Apache redirect so non-www URLs went to www instead. Lines were added to .htaccess file.
    • Changed the color of the text in mChat when editing chat text. It was white on white, made it black on white.
    • Added a favicon for the site
    • Installed an extension to open external links in new tabs
    • Installed the Profile Flair extension. This allows badges to be created.
    • Installed the YouTube Video Gallery extension. It requires creating an application in Google console to enable, which the client did.
    • Updated phpBB 3.2.1 to 3.2.2. Applied three manual changes afterward, bugs that were subsequently found in the release.
    • Edited some colors on the portal sidebar that kept text from being seen.
    • Fixed a resizing issue with embedded videos on mobile devices by adding max-width:100% and max-width:100% CSS directives for the object tag.
    • Made more tweaks to colors for the “time” line near the navigation bar, made better CSS selectors for poll links in the portal and moved vote buttons in the portal more to the right, along with vote totals.
    • Radio extension turned out not to be viable. So I disabled it. I tried to place the player manually. I tried a number of approaches, once I figured out how to write the Javascript correctly. The easiest way was to turn it into an ad in the Advertisement Management extension and place it above the navigation bar.
    • More trouble with Tapatalk when the client tried to add a permanent Tapatalk key. Again the extension failed in an ugly manner and again the issue was some sort of database privilege that should have been there was not, probably for creating new objects like tables, columns and indexes. I commented out some code in the extension to get it to come up so the board was usable, but feels jury rigged. Installed some smilies that client needs to configure. The client is getting with his web host to try to solve the underlying issues of database privileges needed to create new tables that seems to be the issue with Tapatalk.
  • I upgraded forum from phpBB 3.0.10 to 3.2.2. I applied patches to 3.2.2 after the upgrade. There were two issues during the upgrade. The first (duplicate row in config table) went away by running the upgrade again. The second required using phpMyAdmin to edit the prosilver style and make it active. Then the upgraded completed. I removed some dead modules carried over from phpBB 3.0. I created keys on reCaptcha site and entered them into reCaptcha spambot settings, but they were not recognized. I eventually discovered if I requested invisible reCaptcha those keys would work. There was a similar issue with a Cleantalk authorization key. The client had to generate a key on Cleantalk site and he installed it himself. I also installed the Advanced BBCode extension. I noted an email error I encountered with the upgrade. It may be an issue with integration with email. Later that day when the client removed my FTP account (since it gave access to the web root) it also removed everything in the public_html folder! So his whole web site including the forum was gone! The web host had to recover it. The next day I reapplied the file changes for the upgrade and uploaded the programs for the extensions that were installed. The database was still good so I didn’t need to run the upgrade program again. The client asked me to look into errors in some 3rd party software tied to phpBB in his members folder. It asked for PHP 5.4. I changed his .htaccess file but that exposed an issue that the code was using super globals, so it didn’t work. I suggested contacting the original author and having him update the program to use the new phpBB 3.2 libraries. Client eventually figured it out himself.
  • Lots of work for this client too:
    • I converted a forum from phpBB 2.0.23 to 3.2.2. About 200,000 posts with about 7000 attachments. The conversion went smoothly. Allen Subsilver style installed. I replaced the logo. I recreated search index. I moved the old forum into old_forum folder where it exists as an archive.
    • Later I installed a Board Rules extension and Custom Registration Agreement extension.
    • Customer discovered a weird problem. When using the search index, sometimes a XML parsing error occurred. After lots of puzzling it through I realized I had seen this before. Non UTF-8 characters (probably belonging to the Windows character set) were in post_text column and other columns. The parser could not parse these characters because they were not translatable into UTF-8. I fixed it with SQL using MySQL CAST and CONVERT functions on all columns in tables where users type in text.
  • Removed and re-place some ads in overall_header.html for a phpBB 3.0 forum
  • I got called in as a consultant for a huge forum, the biggest I’ve ever seen: 14M posts! That’s about 4 times as many posts as are on phpbb.com’s forums which have been running since 2001! What a strange issue they were having! For about 48 hours they could operate as normal, then the database would freeze updating num_posts in the phpbb_config table. All they could do is recover the database and start again, then 48 hours later it would happen again. They were very frustrated. The error message suggested that some transaction was not getting completed. In addition, the value for num_posts in the phpbb_config was not consistent with the actual number of posts. On top of all this, this phpBB 3.0 forum has a site Wiki outside of phpBB, and the Wiki made posts to their forum using a MySQL stored procedure. What to do? It took hours to get command line access just so I could look around. Once inside MySQL I looked at the tables. Most were using the MyISAM storage engine, but the phpbb_config table where the problem was happening was using the InnoDB storage engine. If I changed its storage engine to MyISAM, would the problem go away? I did and the problem went away! Oh, I also wrote some SQL so num_posts matched the actual number of posts in the phpbb_posts table.
  • Upgraded a forum from phpBB 3.2.1 to 3.2.2. No surprises.
  • Updated a forum from phpBB 3.2.0 to 3.2.2. A direct 3.2.0 to 3.2.2 update would not work and caused Symfony errors. After much trial and error an upgrade to 3.2.1 then to 3.2.2 worked. I reapplied the logo and styles. I was asked to set permissions so users could edit their own posts and to fix an issue with their posts not going through moderation. It took some puzzling to figure this out. Apparently when a new user registers, they are placed in both the Newly Registered Users group and the Registered Users group. Unless permissions for Newly Registered Users are set to Never the privileges for Registered Users, if more generous, will take precedence. So for Newly Registered Users, I added Never permissions for seeing forums they weren’t supposed to see. Although newly registered users were required to have one post before they were considered registered users, posts are only moderated if the moderation permission is enabled, which it was not. So I turned on these permissions for this group.

Some pricing changes

I’ve upped some of my prices. You can see a list of all my prices on my services page.

I’m trying to recapture the labor involved in answering your questions prior to you hiring me. Not everyone who inquires of my services hires me of course. Typically there is at least an hour of time involved in back and forth emails with clients. This time was not built into my pricing structure, meaning I am effectively discounting my own labor rates by giving away this time for free.

So mostly this amounts to $10 or $20 increases for services I do most of the time, such as updates, upgrades, conversions and extension installations. My basic labor rate is unchanged. Obviously if you choose not to hire me you don’t have to pay anything. But if you do I want the work I do for you to reflect the full amount of my effort. And that typically involves at least an hour of back and forth communications over email, often more, usually over many days, as details about the work are refined.

My prices remain very competitive. If you can find a better deal out there, please go for it.

I also updated the page to discuss developing extensions for clients. In general I don’t do it, but I will refer you to other extension authors I trust. Mostly it’s due to a lack of time as developing extensions takes considerable time. Extension development is quite costly and most clients usually decide they cannot afford to have custom extensions developed anyhow.

January 2018 work summary

I didn’t do much work in January, mainly because I spent most of it on vacation: 19 days in total! It was a great vacation: a 16 day circuit of the Panama Canal on Holland America, from Fort Lauderdale to San Diego with lots of ports of call. In addition we stayed an extra day in San Diego to see its infamous zoo! I do have a few buns in the oven at the moment including a large project for a large client that I am starting work on that may take a month or two.

During my vacation my site’s certificate expired. I tried to fix the issue remotely when I had Internet, but just couldn’t do it with my Windows laptop. That may have cost me some business. It’s all fixed now.

On the infrastructure side, I upgraded my iMac to use a solid state drive. Things are so much spiffier now!

Before and after my vacation though I did squeeze in some jobs:

  • A client got rid of his forum due to low traffic but wanted me to do some WordPress work for him. He was been having WordPress security issues. I examined his WordPress .htaccess file. I found no issues there but redundant duplicate code was removed. I explained how the .htaccess file works and provided advice on updating, finding, fixing and proactively handling security for WordPress including Wordfence and UpdraftPlug plugins.
  • Two forums were upgraded for the same client, both from phpBB 3.0.12 to 3.2.1. Since the forums caters to German speakers, with the upgrade I installed two German language packs (casual and formal honorifics). On the first forum, I installed the prosilver_se style. I installed the Navbar search extension thinking it would help with styling but eventually disabled it as it was not needed. I changed style a bit to somewhat match old logo and background, experimenting with CSS until I found something that fit. Since the user could not login, I created a new account (after enabling registration) and gave myself founder privileges in the database. The next day I upgraded the second forum. I integrating the old logo but it did not look right with the new style. I tried changing the background color for the style but it still looked off. However, it was acceptable to the client. I then installed version 3.2.6 of my digest extension, and set up and tested the system cron job to ensure digests went out in the hour requested. I used curl and set the cron to run at top of the hour. I resolved a permissions issue that allowed any registered user to be a global moderator. There were some hosting issues: I could not find a link to cron in the web host control panel until the customer added it to my account. The next day I fixed a database issue: changed default for user_topic_sortby_type from m to t. This was apparently changed by some phpBB 3.0 mod. I updated the database to change this value to t for all users.
  • Troubleshooting. A forum gave a “is_dynamic missing” error message, which was actually a row in the phpbb config table. It took a bit of puzzling but it turned out that the forum’s config.php file was pointing to a phpBB2 database! No wonder the row was missing! I searched and found another database that looked like it had the tables for phpBB 3. But it was missing the last 5 tables. I found a copy of the database in the store folder dated 12-18-17. With customer’s permission, I reloaded that database. Bringing up board, the phpbb_login_attempts table was missing so I created it manually. I could not login with customer’s credentials so I created an account for myself and gave myself founder privileges via the database. The subsilver2 style was misidentified as Red Silver. I renamed it to subsilver2 in the database and enabled the Red Silver style. This fixed the styling issue and the site’s logo appeared in the header like before.
  • I upgraded a forum from phpBB 3.2.1 to 3.2.2. An issue with quoted characters repeating themselves horizontally and vertically, overwriting the quote, was solved with upgrade. Replicated logo in header and changes to footer. This was my first attempt to update to phpBB 3.2.2. I quickly learned that (a) the incremental update still does not work reliably and (b) you need to apply some patches.¬†Other than that (to quote Monty Python), phpBB was perfectly fine!
  • I upgraded another forum from phpBB 3.2.1 to 3.2.2. Database updater failed due to timeout. The real issue were some bugs in 3.2.2 and had to apply some patches to complete the upgrade. I reapplied the logo. The search index erroneously pointed to nonexistent MySQL index. Pointed it to phpBB native and message went away.

December 2017 work summary

December was a very slow month. December often is as clients have other priorities as did I. I finished teaching a class and issues two new releases of my digests extension. Revenue-wise, 2017 was my best year so far. Here is a summary of my paid work in December. All client information has been anonymized. Since I will be on vacation for 18 days in January, I expect next month will be slow as well.

  • Undertook a painful upgrade of a forum from phpBB 3.0.1 to 3.2.1. It ended up taking most of the day. A database timeout meant I could not upgrade phpBB on the client’s site. So I had to extract the database, download it, recreate it on my machine and convert it there. Some tables requires multiple extracts to export due to timeouts. Because I have PHP 7 on my main computer and I needed to first upgrade the forum to phpBB 3.0.14 (which does not work on PHP 7), I had to do the conversion on my laptop. After I upgraded the forum on my laptop, extracting the database from the laptop turned out to be an issue until I explicitly told mysqldump to export as UTF 8 (UTF-16 was the default). This seems to be a MySQL peculiarity when it runs on Windows. In addition, the client’s file manager was rudimentary. All files had to move using FTP. Installed the Advertising Management extension and mChat extension but left client to configure. Client will handle styling. Upgraded PHP to 5.6 (was on 5.0).
  • Moved a forum from Westhost hosting to A2 hosting. Client had a temporary domain name on the new host. I first tried to upgrade forum on old server but it didn’t work. I created a new forum on new server but database and files were not moved by the new host. Client provided a copy of the database. I loaded it on local machine (3.1.9 database) and upgraded it there to 3.2.1. I then created a new forum database on new host. I populated it with upgraded database from my machine. I changed folder permissions to standard. Moved /files, /images and /store folders from old server. Changed Imagemagick path as it had changed so embedded images would display. Installed ID_Launch Fresh Style. Installed Google Analytics and Shareon extensions.
  • Installed Authorized for URLs extension. I went through the client’s numerous user groups and disabled privilege for bots, guests and newly registered users. I enabled extension to work for every other group. I provided instructions for its use.
  • Troubleshooting. Slow server, mostly manifested in slow database but also a lot of spam traffic. I discovered a WordPress temporary table issue. Two search tables in phpBB had repair issues as well; repairs would not stick for long. To keep a lot of the spam traffic from ever reaching the server, I installed Tapatalk both for WordPress and phpBB and enabled its spam firewall feature. I changed the storage engine for forum tables to use InnoDB. Converting the search tables to InnoDB required doing it from the command line as phpMyAdmin would time out.
  • Issue with routing and mChat on other than first page of the mChat display. Not sure what was causing issue is. .htaccess file for forum was missing uploaded it but didn’t solve the issue. Suggested disabling mod_rewrite. Installed SEO Sitemap extension and Italian language translation for it.

November 2017 work summary

Business here at phpBB Services was a bit slow in November but it picked up toward the end of the month. It worked out to a nicely profitable month in the end. Here is a summary of the work done this month. All identifying client information has been withheld, of course:

  • Tried to fix more malware issues for a client who uses WordPress for a site to advertise his business. Because of the malware, his web host would not allow me access to the site to fix it, but they did provide a Dropbox with the files and identified the known files with vulnerabilities. Setting it up on my machine was complicated but I finally got it to work. I corrected the known problems and compared WordPress and all plugins with referenced files. I checked the database too for malware. I removed all unused themes, including one with a known vulnerability. I updated a few plugins and WordPress itself. I zipped everything up and sent it to the web host, who installed it and found no errors. But later on despite my best efforts and scanning all the files by the web host, the malware returned. The malware looked like it has been injected into a business theme. The frustrated client ditched WordPress. Not sure what he is using now but I think he moved to weebly.com. I can’t say I blame him. I tried to fix this for him as a favor. While I dabble in WordPress but don’t consider it an area where I have any particular expertise.
  • I installed the Advertising Management Extension on a forum running phpBB 3.2.1. I spent extra time troubleshooting and analyzing his ad placement questions … apparently there was an ad missing in the footer. I found one missing on the portal page and suggested checking his Adsense code to see if it is still correct, as it left a blank area on the page.
  • I upgraded a forum from phpBB 3.1.9 to 3.2.1. Since the client has a SCCS-based style, I reinstalled the Artodia SCCS compiler extension after the upgrade so I could give it the desired look. The colorizeit.com site unfortunately doesn’t work on phpBB 3.2 styles, so I looked at changes made originally using colorizeit.com to his Elegance style for 3.1 and replicated them for his Elegance style for phpBB 3.2, creating new custom style I called Elegance Orange. There were some issues with drop down controls and the style that the client noticed. Fortunately, I found a newer version maintained by another style artist that fixed these bugs. I installed that version of Elegance, made a new Elegance Orange style again and reapplied the code changes to the theme and templates. Customer satisfied!
  • Client had issues accessing tabs in the Administration Control Panel. I tried to look in the database using phpMyAdmin but got a token error trying to run phpMyAdmin. I figured the issue was with the database itself, so I repaired it outside of phpMyAdmin. There were still issues so I deleted the cache manually and the errors stopped. The client approved converting tables to InnoDB to reduce likelihood of further issues. I had to put Unix 777 permissions on the cache/production folder. For some reason they did not inherit. I recommended that the client complain to web host and ask to be moved to newer hardware and software as he was constantly running out of quota and they were using old Windows hosting.
  • I updated the Tapatalk extension from version 2.0.3 to 2.0.5 based on email client received from Tapatalk.
  • I moved a forum from FastComet hosting to TMD hosting, both the database and files including files for the website, not just for the forum. I changed the DNS in GoDaddy to point to the new hosting.
  • I updated the Tapatalk extension from version 2.0.3 to 2.0.5 for another client.
  • I installed the Advertisement Management Extension on a phpBB 3.2.1 forum using the standard prosilver style.
  • I installed my digests extension (version 3.2.4) on a new phpBB forum. There were some issues getting the cron to work right. It required a support ticket to the web host to get path to curl.
  • I updated a forum from phpBB 3.2.0 to 3.2.1. I had to upload all the files rather than the changed files because the upgrade would not work otherwise.
  • I upgraded forum from phpBB 3.0.10 to 3.2.1. Due to the high availability requirement of the forum, it was a two step process, first to verify it could be done at all. In the first step, I created a test database and copied the production database into it. This exposed MySQL shared hosting quota issues, so it took a while to copy all the tables correctly. Trying to recreate the search index on the test forum also caused MySQL quota issues. I installed the Flat style and integrated the old logo, changing some CSS to place the transparent logo image on a black background like before. Converting the production forum was straightforward. I placed a copy of files and database into a backup folder first. I copied theme as I configured it from test forum to save time. It all looked good so I deleted old database, the test forum folder and subdomain too.
  • Client worked with his web host to move his WordPress site and phpBB forum to a new server. The forum did not come up. Upon investigation, I determined that the forum’s files were moved but database hadn’t been moved. I exported the database from the old hosting and imported it into a new database on the new hosting. I edited the config.php file so it correctly connected with the new database. I purged cache manually otherwise the cached file would still try to connect to the old database. I set folder permissions correctly. I then upgraded the forum from phpBB 3.1.9 to 3.2.1. There were no complications, just one image to replace. Later the user discovered that post images weren’t showing. When I tried to bring up the forum but the phpBB3 folder was gone! I recovered it using File Manager. The post images still didn’t show. It turned out that the path to Imagemagick had changed from the old machine. Once the path to it was fixed the images showed. I also changed some settings to use the new security certificate installed.
  • I fixed a weird issue that developed with Forumrunner modification installed on a forum running phpBB 3.0.10. Forumrunner is similar to Tapatalk in that it allows a forum’s posts to be seen inside of a smartphone app. In the forum’s error log, I discovered an error that said the Forumrunner program expected to write to a missing fr_token column in a table. I scoured the web for a reference and found its datatype (tinyint), then added the column to the table with phpMyAdmin. It should make this issue go away. I also provided an estimate for upgrading to phpBB 3.2.1 but the client is not interested at this time. Hopefully there are no more Forumrunner issues like this!