Month: October 2015

I’ve been doing this phpBB consulting work part time for more than nine years now. Lately I’ve learned a painful lesson that you can never be too careful.

A client with a 1.4M posts database wanted to upgrade from 3.0 to 3.1. I dutifully did what I always do before starting any work: backing up the files and the database.

The files are straightforward to backup. The database on the other hand…

Shared or VPS  hosting have some disadvantages: resource limitations that are not always known. It’s more problematic with shared hosting. phpBB admins suffer through these all the time. Database backups don’t finish because of HTTP 500 timeouts, for example. That’s what happened with this client. Not too surprising that with 1.4M posts, the phpBB backup would time out because shared hosting can’t create a backup in the number of CPU seconds that a program will get from the host, allocated in Apache and PHP’s configuration.

So I went into the web host control panel, kicked up phpMyAdmin and used its export utility. With 1.4M posts, the download took a long time, 15 minutes or more. But it downloaded, or at least appeared to download successfully. There it was in my downloads folder, and ungzipped it was 2.4G. I made a number of these over the several days I was working on the project as I tried a number of approaches to upgrade the forum. Kept getting mysterious errors documented here.

No problem if the conversion failed because I had both a file and a database backup. Of course, reloading 1.4M posts and associated data takes a while. First I had to drop the tables. I ended up using a staggered MySQL importer to import everything. It took hours.

But after all that effort when I tried to bring up the board, it complained about missing tables.

I finally looked at the bottom of the database export using the Unix tail command. The export was not complete. It had timed out, but there was no symptom that it had. All my other exports had failed for the same reason, so none of my backups were complete, and there was no clue to let me know something went wrong.

So I’ve spent the better part of a week trying to bring it back. We were at the mercy of the web host, who had some stuff backed up, but inconsistently. Some tables had newer tables than others. Fortunately the posts table looked complete and a copy of the users table looked complete. I wrote a program to recreate the missing topics. So things are back, sort of. Client is not happy but under the circumstances he was lucky to get it back at all. And I encountered an issue I never had before and learned a painful lesson.

It’s for these reasons that I’ve revised my pricing to up my fees on boards with lots of posts and users. This work becomes unprofitable at a “one size fits all” rate. There are too many minefields. Now I have to verify that my backups are actually complete. I can’t rely on software to do its job.

• Created an updated version of rss.php originally written for phpBB 2 by some other developer that would work with phpBB 3.1. This allowed the client to seamlessly integrate recent topic titles into dynamic content areas on his other websites and elsewhere on the domain hosting the forum.
• Upgraded forum from 3.0.7-PL1 to 3.1.5. Had to change some table columns collation types to UTF-8 and install en_us language pack to complete upgrade. Reinstalled logo. I then spent two hours systematically finding all BLOB columns in his database and changing these to the reference implementation for MySQL, mostly VARCHAR and TEXT. I also went through all tables and looked for columns where the collation was not UTF8-BIN and changed it to this. This should address any future issues with inserts failing due to no default values for a column. Note: there were subsequent issues with default values. Went through all the tables comparing schemas with a reference and adding SQL to give default values to columns where they were missing. Also checked indexes and fixed a few when there were concerns about performance. Changed all tables to use the INNODB engine.
• Upgraded a forum to 3.1.5 from 3.0.12. Assisted with malware removal by removing two files identified by the SiteLock software ($15). Fixed an issue with getting into Administration Control Panel resulting in a HTTP 500 error. The problem was because the board start date was not formatted in the database as a Unix timestamp.
• Added 600GB to a volume on a Linux dedicated server that had ran out of quota.
• Digests cron was deleted resulting in no digests going out hourly. Made various attempts to create a new cron that successfully sent digests but nothing worked. Got it to work from my uptimerobot.com account. User then created their own account on uptimerobot.com. Created the monitor there and removed it from my monitor.
• Worked to add the look of AcidTech Green (a style for phpBB 3.0) to a client’s phpBB 3.1 forum. Decided on using a 3.1 DVGFX style, which is prosilver based. Reused the old logo, but it took a while to figure how to center it. Had to change HTML in overall_header. Integrated the menu from the web site main page into the forum. Adjusted some colors for forum titles, topic titles and post titles. Waiting on FTP permissions for the main site so I can replicate the new forum logo on other static web pages.
• Upgrade forum from 3.1.5 to 3.1.6.
• People could not log into forum. Mangled front end splash page contributed to problem. Created 3.1.3 instance in a /forum folder and attached it to his database. Updated links on splash page for contact page, member list and member login. Changed permissions so guests could see memberlist to replicate old functionality that did not work. Previous developer developed a 3.1 style that had many customizations on it, based on Artodia Elegance Black, but seems to be lost. Client is trying to recover it so I can replicate the changes to the style in the new /forum folder. Only 3.0 styles were found in the recently recovered website folder.
• Upgraded forum from 3.1.5 to 3.1.6. Still dealing with some registration issues of new members being treated as spam users. Need more info on that.
• Client upgraded board to 3.0.14 and users could not register. Version looked old. Upgraded him to 3.0.14 and reapplied Board Portal 2.0.2 and the abandoned Dicemod 8.8.8b and it came up and I could register.
• Work for September for an existing client. Worked to define requirements and find a developer for an extension that client would like developed. Participated in a Skype conference call with client to discuss various issues. Installed a larger 32×32 favicon at client’s request. Discussed viability of a phpBB access form with client. Upgraded forum from 3.1.5 to 3.1.6. Participated in discussion with client regarding concern about inability to drop files into phpBB. Discussed issue of potential bug with Topic Preview extension with client. Converted phpbb_config table to expand config_value column to varchar(8000) to address probable bug that occurred when a client added attachments to a post. That did not solve the problem so I did more debugging. The real problem was that an explicit SQL CAST statement was needed in phpBB for SQLServer. Created a patch to /phpbb/config/db.php to address phpBB bug and filed a bug report on phpbb.com.