Month: May 2018

Cleantalk extension for phpBB can remove spam posts, plus its spam firewall feature is very useful

On: By: Mark

This is an update on an earlier post on removing spam posts.

Removing spam posts is hard because it requires actually reading the post and deciding if the post is spam or not and then using moderator tools to remove these posts. If your forum is overwhelmed with spam posts, this is a Herculean endeavor. Ideally though posts could be “read” by software and it would make the judgment on whether it is spam or not.

The Cleantalk extension for phpBB 3.1.x and 3.2.x can do just this as well as lots of other really cool tricks. My customers love Cleantalk, but the service is not free. However, it is so inexpensive that it easily justifies spending $8/year for the service. You can subscribe on the Cleantalk website. As of this writing, you can try it for free for 7 days. After 7 days, it won’t bring down your forum but it will stop working.

What is Cleantalk?

Cleantalk is essentially a huge database of addresses of known spammer sites. While it’s not perfect, based on the experience of my clients it is about 99% perfect. I originally recommended it as a spam registration solution for my clients. It still does that but is less necessary since phpBB 3.2. This is because since phpBB 3.2, version 2 of Google’s reCaptcha is supported. Unless it gets hacked, as long as you have it properly configured as a spambot countermeasure it should prevent virtually all spam registrations.

However, it has two powerful features that still keep it relevant for phpBB forums.

Cleantalk ACP Interface
Cleantalk ACP Interface

Installing and enabling Cleantalk

Cleantalk is installed like any other extension. While it can be downloaded from phpbb.com, you should download it from its GitHub page. This is because as of this writing the version on phpbb.com does not include the spam firewall feature, and you will probably want to enable this feature. You can access it through the Administration Control Panel: ACP > Extensions > Antispam by Cleantalk. Before you can do much with it you have to enter your Cleantalk key which you can get from their website or by pressing the button in the extension that should retrieve it for you.

Removing spam users and spam posts

As you can see from the image, once the extension is enabled and the key is properly configured there is a prominent Check users for spam button on its page within the Administration Control Panel. If you have lots of users, it may hang. Based on my experience though the next time you go into its interface you will see a list of potential spammers.

As I said, it is not perfect. So I recommend that for users with posts to check these out these users topics to make sure their posts are spam before deleting them. For those you want to delete, check the boxes next to their usernames and then press Delete marked. You can also press Delete all to remove all users and their posts. You may have to go through many pages to delete all spam users and their posts, but this is obviously much faster than doing a visual inspection of all your posts.

Spam firewall

This is a new feature which as of this writing is not available if you download the extension from phpbb.com. It keeps almost all spammers from hitting your site at all. Instead, Cleantalk’s servers grab it first. In the event the user is legitimate, there is a link that will take them to your website.

Why is this useful? Because it reduces the stress on your server by limiting it to legitimate traffic only. It speeds up the performance of your forum and makes it less likely that you will have to pay for the cost of a higher class of hosting to handle your traffic. Isn’t that worth $8 a year?

Stopping contact form spam

Cleantalk has one other useful feature: the ability to stop contact form spam. Of course you can disable the contact form (ACP > General > Contact page settings) and that will solve that issue. Or you can have Cleantalk essentially moderate it for you, passing on only valid contact forms to you. Simply check that option on the extension’s page and submit the form. Somewhat oddly, the phpBB group did not tie the contact form to the spambot countermeasure feature of phpBB. Perhaps that will come in a future release.

In any event for forums that get lots of spam and/or lots of traffic, using the Cleantalk service with the Cleantalk extension for phpBB is a no-brainer providing you know about it. Now you do!

MediaTemple grid service no longer recommended

On: By: Mark

It’s sad for me to say this, but I can no longer recommend MediaTemple’s Grid service as a hosting option. For the last 18 months or so I’ve been using this service and have gotten increasing dissatisfied and exasperated by it. Today I started the process of moving my domains off of it to Siteground.com even though I have four and a half months left on that hosting contract. This domain should now be coming to you from siteground.com servers and hopefully in a reliable and maybe spiffy fashion.

MediaTemple.net was known as one of the premier providers of business-class hosting. It was acquired by GoDaddy with the promise that it would be separately managed. It appears from my experience with their Grid service that they broke that promise with their customers. Sometimes I have to wait a minute or more to retrieve pages from my own site. It sure looks like they are overloading their servers and/or managing them very badly.

I used UpTime Robot to test whether my domains are up. Pretty much every day I will get one or more emails telling me it is inaccessible. So it was likely costing me money, motivating me to move to Siteground.com instead.

I do have clients using MediaTemple’s virtual servers and they have no complaints about that service so far. Definitely avoid their Grid service now and if you have an option you might choose some other host for your virtual or private server needs. I don’t have experience with Siteground’s, but it’s likely fine. A company like Rackspace.com is likely doing it right.

Another annoyance was revealed simply in moving my site. An old phpBB forum I have with about 50,000 posts could not be downloaded. I had to break it down into multiple downloads, including the posts table into two separate downloads. The Grid service simply cut me off when I hit some sort of resource limitation. The whole database is only 80MB or so. Shame!

April 2018 work summary

On: By: Mark

In contrast to March, April started out busy and ended up slower. This is in part because much of my March work continued into April. Anyhow, here’s what I worked on for clients in April. As always, I do not identify my clients by name or domain unless they specifically allow it:

  • I spent a lot of time continuing work for a client I started working for in March. Their WordPress site integrates phpBB into a separate area.
    • First I completed the move of the forum from phpBB 3.0 to 3.2. The hard part here was making changes to the phpBB style to integrate the overlaying WordPress theme in the headers and footers, as there were many CSS class and HTML ID conflicts.
    • As part of the upgrade, the client took the plunge and moved the site from standalone web pages to WordPress. A lot of scripts written by someone else to monetize particular types of posts broke because of changes to phpBB’s API. These old programs were “tightly coupled”, to use a software engineering term. Basically as these broke I fixed them by updating the phpBB code inside these programs. Most of this work involved changing $_SERVER, $_GET and $_POST statements to use phpBB’s request object instead. 
    • With the new WordPress framework, some of the old content appearing in sidebars needed to move into WordPress widgets in the sidebars of the theme. We’re taking this piecemeal, learning as we go along. A better “loosely coupled” way to get phpBB content in WordPress is to use phpBB feeds. phpBB comes with a built in Atom feed that needs to only be enabled. But it doesn’t do everything to retrieve relevant posts. WordPress has a feed widget and there are also a number of other widgets available as plugins that do a fancier job of serving feed content from Atom or RSS feeds inside of WordPress. In the case of my client, the sidebar needed to show recently posted topics in the forum. I eventually decided I needed something beyond phpBB’s Atom feed, so I chose my own Smartfeed extension instead. However, it didn’t return quite what the client wanted as he only wanted “fresh” topics from a certain forum, and only the latest post from these topics. I was able to tweak my Smartfeed extension to do this, and this inspired me to work on a new release of Smartfeed to add these features. Once deployed, I will update the version of Smartfeed I installed for this client to use it instead.
    • I fixed an issue with moderator permissions with the forum by adding these moderators to a list of global moderators.
    • Installed a SiteMap extension so posts were better available to search engines.
  • Upgraded a forum from phpBB 3.2.0 to phpBB 3.2.2. Replaced the logo in the original dimensions. Upgraded Cleantalk to latest version on GitHub. Enabled the spam firewall feature of Cleantalk. Installed announcement on index extension for 3.2 (not official). Fixed issue with wrong reCaptcha. Checked global moderator privileges and they were okay. Administrator moderator role was changed to Full Moderator.
  • Upgraded Tapatalk extension to version 2.0.8. This also meant removing the mobique directory and reuploading that.
  • Changed the people who have moderator privileges for a client.
  • This work was a bit different. I created two MySQL triggers so if a user is added to a particular group by an administrator they are subscribed to receive notifications if new topics are created in a forum. I also created a trigger to remove the forum notification if they are removed from the group.
  • A client needed URLs embedded in the post text changed as the image hosting service has moved it to a new domains (postimg.cc and postimages.cc). It’s fairly simple to do once I finally figured out about the MySQL replace function. I also installed the Member Profile Views extension (1.0.3) for the client. Later, I updated the Cleantalk extension to the latest version. While I was in there removed 3 dead extensions from the extensions table that were showing as ugly error messages on the Install extension screen. I also help create a new BBCode to serve embedded videos from a site and updated the latest version of phpBB Gallery from the GitHub master: version 3.2.2.
  • Upgraded a forum from phpBB 3.0.11 to phpBB 3.2.2. Installed Allan Style Subsilver. Placed a new logo. Installed Advertisement Management extension and placed two old ads, changing the image URL a bit. The forum has 200,000 posts. One minor issue during upgrade: I had to make active the prosilver style for it to complete.
  • Completed a messy upgrade from phpBB 3.0.12 to 3.2.2 on GoDaddy hosting. The file folder was missing and was apparently dropped a month or so back when user went over quota, and was not recoverable. This meant lots of images embedded in posts would not show. The user has to live with this. I had to move the database to my machine to convert it, due to severe resource limitations on GoDaddy. The phpBB native search index creation did not work, so I created a MySQL Full Index one instead. mChat was previously installed as a mod. I had to manually remove all traces of it to get the mChat extension to install. Many tables were missing primary keys, indexes and automatic increment settings. I eventually figured out this was an issue with bigdump.php, a staggered MySQL importer I used, as it nears the end of a database extract file sometimes and hangs. These are stored at the bottom of the file and it never said it completed. It took an hour or so to create the missing indexes and primary keys. I did not move over tables for the search index, topic tracking information, the session information or the logs due to size issues. I could not find a way to remove spam posts easily but did remove old inactive users. Installed reCaptcha V2. Installed Tapatalk, Advertisement Management (to place ads at top and bottom), ShareOn, mChat (after cleanup) and OneAll Social Login extensions. Adjusted forum permissions so newly registered users could post without permission and gave full access permissions to all forums for all major groups.
  • As with the first client, I changed some URLs in the post text to use postimg.cc and postimages.cc. This one though was a bit different because periods were stored as HTML entities in the database. I also spent an hour trying to make the upload attachments feature work since he did not want to use external services anymore for posting images and he had plenty of space. There were lots of permissions issues to puzzle through. I had to change forum access privileges for registered users to full access. I also had to set the upload permission for the registered users to allow it and change the quota for uploaded images from just 1MB to 10GB.