Cloud service types and phpBB

In a recent post I looked into putting phpBB in the cloud, the first in a series of related posts. In the first post I said that while it is possible to do it, phpBB is not a cloud-first software solution. There are advantages to putting phpBB in the cloud, such as scalability and potentially lower costs. But there are significant drawbacks for most forum administrators too, including almost no technical support and you must provide any system administration.

Cloud service are arranged around various service types. While the types can be breathtaking at times (look at Amazon’s many specialized web services) you can sort of lasso these into three types: IAAS (Infrastructure as a Service), PAAS (Platform as a Service) and SAAS (Software as a Service). As you will see, phpBB doesn’t fit neatly into any of these models.

IAAS (Infrastructure as a Service)

With IAAS, the cloud provider provides you with a set of basic tools and lets you have at it. The tools can be refined somewhat but generally you get an operating system (usually some variant of Linux), a web server (generally Apache) and a database (generally mySQL). The actual set of tools that you get depends on the package you select. In Amazon Web Services speak, this is the AMI (Amazon Machine Instance). Yes, there are AMIs for phpBB such as intuz’s and BitNami’s. However, these packages are not necessarily free. BitNami, for example, creates a “smart” integration of these tools along with other ones that are optimized for various cloud deployment patterns. You will probably be asked to pay for the privilege. A good package though it well thought out and cleverly integrated for maximum functionality and performance, and these phpBB AMIs should be finely tuned for phpBB. You might want to use one of these packages for an upfront cost and then rent it for a monthly fee.

Some packages/AMIs are free and come from a common library provided by the cloud vendor. You might want to install one of these instead. They likely won’t come as well integrated, but they will get the job done.

As much as an intelligent package/AMI helps, you still don’t get much in the way of handholding. You will generally get the most basic of control panels provided by the cloud provider and some SSH and database credentials. You are expected to know how to use SSH, FTP and relational databases. Of course as time goes on, your operating system, web server and database software will need updating. For the most part you will be expected to figure out how to do this. You will also have to worry about things like malware getting onto your site and deal with any technical issues that come up.

If none of this intimidates you, IAAS is a great choice and probably cheaper than using a web host. But basically you are trading your own time and expertise to lower hosting costs, but as a bonus you should get a scalable cloud service to handle high and low demand periods, that is if you set it up right.

PAAS (Platform as a Service)

In general, PAAS cloud services are oriented around developers. If you want to be the next Facebook, for example, PAAS will provide you not just with infrastructure as a service but also a set of development tools finely optimized for you to develop unique cloud services. They tend to be oriented around one or more programming languages, such as Java or PHP, and one or more deployment engines, such as Kubernetes. The nice thing about PAAS is that you don’t manage much. Should the Java J2EE engine need an update, that will be handled for you. Consequently, PAAS tends to cost more than IAAS. You and your development team are then free to focus on developing that next Facebook.

phpBB of course is a prepackaged software solution. You shouldn’t be developing anything, unless perhaps you are an extension developer like me. But if you are an extension developer like me, you don’t need to develop your extension using PAAS because phpBB is not typically deployed in the cloud. Instead, you have a local web development environment and do your development there. You might want to test it on a web server connected to the Internet, but you don’t need PAAS to do that.

So basically, using the PAAS cloud service type for phpBB makes no sense.

SAAS (Software as a Service)

In the SAAS model, you use a cloud provider to provide a software solution in the cloud. Perhaps the best known SAAS provider is Office 365. Why install Microsoft Office on a PC when you can run it as a cloud service instead? It’s a popular model that is making some companies like Adobe (and its Adobe Creative Suite) or Salesforce.com tons of money. They’ve already figured out an optimal solution and they are happy to rent it to you for a fixed cost per month. If you don’t need it anymore, you cancel your contract. In some cases, you do actually download some software on your PC that works optimally with these services in the cloud. Many developers using PAAS are actually hoping to market their solutions as SAAS. Since SAAS is deployed in the cloud, it is presumably finely engineered for optimal performance in all kinds of workloads.

The thing is, you can sort of get phpBB as SAAS already. phpBB has a knowledge base article on how to do it with Microsoft Azure. There are also a number of sites that allow you to create phpBB forums on their servers, which are often for free for low usage sites, sort of how you can host a blog on wordpress.com for free under a subdomain like myblog.wordpress.com. And if you have web hosting already, there is usually a scripting center that allows you to install phpBB. But are these really providing software as a service? They arguably don’t because once you install phpBB you generally don’t want to use it “out of the box”. You want to change the style, or add extensions, or do all sort of fine-tuning. If you don’t need to do these things, then maybe these “SAAS” services are what you need. They just aren’t really SAAS, since you can’t give your site this degree of customization, because there is no way to do this other than to use phpBB. And if you use phpBB you will need to be able to upload and edit files on your phpBB instance.

Conclusion

With a better understanding of the cloud service types, you should now understand why phpBB is so rarely placed in the cloud. At its root, it’s because phpBB is not a cloud-first product, and probably never will be. In a future post, I’ll look into deploying phpBB using IAAS to give you some idea of what you might be getting into.

 

Should I host phpBB in the cloud?

So I’ve been playing with cloud providers, most recently looking at the Google Cloud in context with a WordPress group that I belong to. But over the years I’ve also studied Amazon Web Services, the original cloud provider. There are other cloud providers but really the only other major player is Microsoft Azure. Host on another cloud and you may find out that it won’t last in the long term, or is not a real cloud service. My goal is to eventually demonstrate how to run phpBB in the cloud, starting with the Google Cloud that I am currently exploring, and bring you along for the ride.

Characteristics of cloud services

There are lots of definitions of cloud services and cloud computing. From the perspective of someone who owns and manages a forum, all you probably really want to know about it what makes these services different than your typical web host like Siteground or GoDaddy. You will get lots of answers. Last year I helped move a big forum to Amazon’s EC2 cloud service. I got some preliminary answer from that work. Some differences:

  • Cloud services are scalable. If you have a host like GoDaddy and you outgrow the resources you are allowed to use inside the scope of your contract, you will get a little leeway. But generally you will be asked to move up to a higher class of hosting. This contrasts with cloud services. Its architecture lets your site grow seamlessly, at least if you set it up right, scaling up by a factor of 100 or more as needed, and maybe dropping back down after demand eases. So if you expect to have a forum that will get suddenly very, very popular, hosting with a cloud service should be a big selling point. Whereas, moving to a higher class of hosting at GoDaddy is potentially a lot of hassle.
  • You don’t get handholding with cloud services. Don’t expect to have a support hotline or a phone number that you can call to reach someone to help you struggle with technical issues. You either brings these skills with you or pay someone to leverage them for you.
  • Cloud services are not for cheapskates. It’s not that cloud services are inherently more expensive than traditional hosts. In many cases, cloud hosting is the better buy because you pay for what you use. Cloud services are elastic to scale on demand, so sometimes your costs will go way up for a given month. That’s because you are getting a lot more traffic or are using a lot more space or require additional virtual CPUs. Very tech savvy people who have small sites might be able to host for free in “micro instances” of these cloud services. Here’s a video that shows how a tech savvy person can spin up a site on a cloud service for less than a dollar a month, providing you know your site’s usage will be minimal. But you really have to know what you are doing. 

Why to not use a cloud service

  • You’re not a techie and want to stay that way. If you are paying $20 a month to Siteground, for example, it will seem like a rip off if you can do the same thing for $1 a month on the Google Cloud and it fits in a micro instance. But in most cases, it’s not. With web hosts, someone else is managing the infrastructure, providing 24/7 support and they provide a host of tools like cPanel to easily do things like manage files, create backups and create email addresses. You are free to concentrate more on what matters: your site and its content, and leave the heavy lifting to a company which is probably doing it for hundreds of people using the same machine you are. Someone else worries about security patches, system upgrades and site vulnerabilities. Yes, often if you manage a phpBB forum, you do have to put your hands into the soil, so to speak. You might have to create email addresses or tweak something in a database, generally in a control panel like cPanel or Plesk. But that’s a whole lot easier than trying to upgrade a Linux kernel or managing an email server’s firewall rules.
  • You prefer fixed costs. You don’t like surprises, particularly financial surprises. With a contract and a good web host, you know what you are paying for, for how long, and what you can expect while you host with the provider.
  • You don’t need to worry about your site getting quickly popular. Your forum is not the next Instagram. It may grow some during the year and there may be some spikes in traffic here and there, but it’s manageable.
  • You like having tech support on speed dial. Hosts of course vary in the quality of the technical support they provide, but knowing you can call a technical person on the phone or chat with them online about some weird problem you are having is comforting.

With phpBB in particular, while it can be made to work in the cloud, it is not a cloud-first product. In truth, cloud-services are mostly for developers and large organizations. Generally they want the reliability and high “up time” that cloud services provide. Organizations use cloud services to mitigate their risks and lower costs; maintaining their own servers and technical support staff is expensive. Most of these organizations though do have developers. They are writing or maintaining systems or services to handle lots of needs, and most of these are proprietary, not using off the shelf software like phpBB. The exception is the Software as a Service model. Some companies like salesforce.com offer their solutions as services you can rent, and put their services in either a public or a private cloud of their own.

In my next post, I’ll look at the other two models cloud services offer, Platform as a Service and Infrastructure as a Service and explain why neither is a great match for phpBB. That said, sometimes you might want to put phpBB on one of these service types anyhow. We’ll explore why and the tradeoffs involved.

Solving phpBB forum permission issues

I have noted before that phpBB’s permission system is awesome. In one way though it’s a bit defective: it’s hard to troubleshoot issues with permissions, particularly forum permissions.

In this post, I’ll delve into solving forum permission issues. The general problem is that a user typically belongs to more than one group and different groups can have different forum permissions. If you are unfamiliar with the basic phpBB groups, you might want to read this post first. You can also create groups of your own and set forum permissions to those groups.

To solve these issues, you generally you need to start with a test case. This part at least is pretty easy because you usually have a user whose permissions are not working correctly, so you just need their username. You also need one or more forums where their permissions are off. You also need to know what permission isn’t working right, such as permissions to create new topics or reply to topics.

If you dig into phpBB’s documentation, it becomes a little clearer. On permissions, the documentation says:

  • YES will allow a permission setting unless it is overwritten by a NEVER.
  • NO will disallow a permission setting unless it is overwritten by a YES.
  • NEVER will completely disallow a permission setting for a user. It cannot be overwritten by a YES.

So essentially when the NEVER permission is set, it becomes a blocker overriding any other permissions.

Seeing all forum permissions for a user, forum and permission type

How do you see these forum permissions? You need a tool. The good news is that phpBB has just such a tool. The bad news is that they bury it. In fact, it could not be harder to find. In this example, I will use my development forum to see forum permissions for myself.

  1. Go into the Administration Control Panel
  2. Select the Permissions tab
  3. On the left sidebar, go way down to the bottom. You want View forum-based permissions. Click the link.
  4. Pick the forum or forums you want. In this example, I chose the “Your first forum” forum created by default when you create your board. I then pressed SUBMIT.
  5. Now I pick the user. There are various ways to do this with the interface for both users and groups. In this case I choose myself, with a username of “Mark D Hamill”. I entered the username in the “Find a member” field and pressed the View permissions button below the field. This brings up the Viewing permissions page. The colors you see on the permission tabs may vary from mine where the green boxes in each of the tabs basically say “This user or group has YES for ALL permissions under this tab”. Red means “This user or group has NEVER set for all permissions under this tab”. Blue means “This user or group has a mixture of permissions for the permissions in this tab.”
View permissions page
View permissions page
  1. I now want to check out a particular permission. You may have to hunt for the permission you want to check for the user and forum as it may be on a different tab. There is a tiny little icon to the left of each forum permission. That’s what you need to click on. In this case I want to see how permissions are determined for the Can start new topics permission.
Selecting a permission to check
Selecting a permission to check
  1. This generally brings up a popup window. If you don’t see a popup window, you may have to tell your browser to allow popups for the domain. Finally this brings up a useful screen:
Viewing a particular permission
Viewing a particular permission

Now you can see what’s going on for this particular user’s forum permission. Since I am both an administrator and a global moderator, I belong to both those groups, each of which has a group forum permissions too. By default, NO access is allowed to the forum for a user, so it is the first permission. But it is set to YES for Administrators, so the logic continues and the next group is tested. It is set to YES for Global moderators too, so the net permission is still YES. It is set to YES for registered users in this forum too, so it’s still YES. Finally, it looks for any user-specific forum permissions. None were granted, so this permission is NO, but since it is NO and not NEVER the overall YES permission still applied.

Fixing the underlying permission issue

With this tool, you should be able to determine where the root of the permissions issue lie from the variance from an expected permission from the actual permission. It’s usually a group permission that needs changing. The most likely solutions are:

  • A NO permission should be NEVER
  • A NEVER permission is blocking everything so it should be changed to NO
  • A NO or NEVER permission is incorrect and should be YES

So adjust the group or user permissions for the forum privilege accordingly. You can use this tool to check to see if the result is correct, or use the feature in the Administration Control Panel to test out a user’s forum permissions.

Usually in these cases, you cannot use the built-in forum roles. Rather you have to click on the Advanced link for each forum and group and change permissions that way instead.

In some cases, you can adjust the permissions for a forum role and have them trickle down accordingly, avoiding the need to use the advanced link when setting forum permissions. I’ll leave you to investigate this option if you want instead. It can get a little hairy to change these because it affects all forums where these permission roles are used.

Setting user-specific forum permissions is always a bad idea. Remove them if you can, and place these forum permissions in groups you create instead. Add people to these groups as necessary to get the desired behavior.

The newly registered users group permission quirk

There are some things that are definitely peculiar about phpBB’s permission system. Newly registered users are also in the registered users group. To start, this makes no sense. In the case of this user, “tester66”, because he is in the newly registered users group, the forum’s permissions for newly registered users does not allow them to start new topics. But because they actually are in the registered users group too, they can start new topics, the exact opposite of what you would expect!

Newly registered users permission quirk
Newly registered users permission quirk

How do you solve this problem? You have to set the permission to NEVER for the newly registered users group by using the group forum permissions function. In it you select the Advanced link to fine tune the permission. After changing the permission, you can see the result:

Fixing the newly registered users group quirk
Fixing the newly registered users group quirk

 

Integrating your phpBB topics and posts into WordPress … or any web page!

As some know, I am the developer of phpBB’s Smartfeed extension. This extension provides an ATOM, RSS1 or RSS2 feed of posts and topics on your website.

These feeds allow people to read posts on the forum remotely using a newsreader application, like one built into MS Outlook or using feed aggregator sites like feedly.com. The main advantage of feeds is that it allows you to read a forum without actually having to visit the site. If you regularly read lots of sites, using a newsreader is very efficient way to read content compared to actually visiting each site.

Smartfeed is not the only extension that does this. In fact, if you don’t need to support the RSS format and only want to show posts in public forums, an ATOM Feed has been built into phpBB since version 3.0.6. You might want to read the knowledge base article to learn the syntax to use. You can control your feeds in the Administration Control Panel: ACP > General tab > Board configuration > Feed settings.

Sometimes you want to highlight recent topics and posts on your larger website, or on another domain. For example, you may have a phpBB forum in a folder on a WordPress site. You might want to use a WordPress widget to highlight current topics and posts on a sidebar of your WordPress site. The links in the sidebar will take people directly to the post or topic of interest.

I will demonstrate how to do this using WordPress. However, conceptually you don’t need WordPress to do this. You just need something that can read an ATOM or RSS feed of your forum, and parse its XML into HTML for display, or write your own program to do this. For example, if you are familiar with jQuery, there are a number of jQuery feed plugins that would work. The PHP SimpleXML library is one way you can do it in PHP with a short PHP program.

Here’s one way to do it in WordPress:

  1. Spend some time figuring out what you want to highlight in WordPress: recent posts or recent topics. I will show a list of recent posts. In this example, I first installed my Smartfeed extension. This is because I got a SimplePie parser error when I used phpBB’s ATOM feed. This is due to a bug when parsing ISO dates in ATOM feeds using the SimplePie library. SimplePie is bundled with WordPress. I reported the bug. The resulting URLs for the feed can be seen if Smartfeed is installed. It is in the HTML source for the forum. I will use the second link because the ?y=2 parameter creates a RSS feed instead of an ATOM feed to avoid the SimplePie bug.
<link rel="alternate" type="application/atom+xml" title="ATOM" href="/phpbb/app.php/smartfeed/feed" />
<link rel="alternate" type="application/rss+xml" title="RSS" href="/phpbb/app.php/smartfeed/feed?y=2" />
  1. Note: if using Smartfeed, and you want a list of recent topics only, the resulting URL will look something like this. You should be logged out when using the Smartfeed interface. Note that you can refine the URL in the Smartfeed user interface. lp=1 limits the feed to the last post in the topic only, t=2 suppresses the time limit for retrieving posts, s=1 gives a standard sort from most recent to least recent, i=0 means to not require a minimum number of words in the post, y=2 forces a RSS2 feed, d=3 sets the feed style to use HTML, w=0 means not to limit the maximum number of words in the post, and tt=1 means to show topic titles only. There are lots of variations so use the Smartfeed interface to get the output just the way you like it.  
http://127.0.0.1/phpbb/app.php/smartfeed/feed?lp=1&t=2&s=1&i=0&y=2&d=3&w=0&tt=1
  1. Presumably you have installed WordPress already. If you haven’t, it can be downloaded from wordpress.org. Or you can usually install it from Plesk or cPanel.
  2. Login to WordPress as an administrator and go to the WordPress dashboard.
  3. WordPress comes with a RSS Widget preinstalled that also can handle ATOM feeds. You can find it off the dashboard: Appearance > Widgets
  4. Click on the RSS Widget then press the Add Widget button which appears, which by default appears on the sidebar.
  5. In the sidebar, click on the RSS Widget that was added. Enter the URL for the feed and give it an optional description. Note that it needs to be the full URL, not the partial one shown above. Then press Save.

    Configuring RSS Widget
    Configuring RSS Widget
  1. I dragged the widget to the top of the sidebar so it would appear first on the sidebar. Of course, you can place it anywhere on the sidebar that you like.
  2. Go to your WordPress site and find it on the sidebar.

    List of recent forum posts in the WordPress sidebar
    List of recent forum posts in the WordPress sidebar

Note that this works for any domain, providing the feed is publicly accessible. So you can promote this approach to have similar sites show your list of recent topics or posts. Note also that my Smartfeed extension has a number of options to make the post subject or topic title prettier if the default looks too wordy.

Fixing insecure content issues in phpBB

Updated December 27, 2018 to correct some things based on new information.

So you’ve decided to use HTTPS for your forum to show your content securely. This is good and it’s not too hard a thing to do in most cases. Everything looks good but sometimes you notice on browsers like Chrome the little green lock icon up on the URL field disappears. What’s going on? If you investigate by clicking on the icon you can usually figure out what’s going on: there is some insecure content on the web page.

What is insecure content?

Insecure content is content embedded on a web page that is delivered insecurely, i.e. from a web server using http instead of https. Usually these come from external sources, and are typically externally hosted images that are served insecurely.

One way to investigate these is to view the HTML source of the web page. Use the Find feature to scan for URLs with http:// instead of https://. The issue occurs with embedded images like this:

<img src="http://www.externalwebsite.com/myavatar.jpg />

If all these URLs could be changed to something like:

<img src="https://www.externalwebsite.com/myavatar.jpg />

then all would be well, that is if the external website supports https.

How do you fix these problems? There are typically two places where these problems manifest:

  • In post text
  • In the user’s avatar

Here are some approaches you can use to solve to fix the problem:

Use the Image Redirect extension

As of this writing the Image Redirect extension is a Beta release, so it is not recommended that you install it on a production system. This extension also requires that you set up a proxy server on your web server, not a trivial tasks and something you may not be able to do on your class of hosting. Camo Proxy is one example of a proxy server you can install. What this extension does is scan the page for these external image URLs, fetches them using a proxy and changes the URL so that it is served from your proxy copy, which will be on your machine and served securely. In theory this extension should solve all issues like this. Note that it takes some time to create a proxy image if it is not cached and this adds some small overhead, which may slow page rendering.

Fix the embedded URLs in your database

This works by changing the URLs in your database. You scan for http:// and replace it with https://. Using this approach has some limitations:

  • The server serving the remote content may not have https installed. What generally happens is the image is not served and a white box appears instead. This could make lots of posts look off or unacceptable, particularly if these images are large.
  • While it corrects existing URLs, it doesn’t prevent someone from doing the same thing in the future.

If you can live with these limitations, you can fix it in the database. This approach assumes you have MySQL or MariaDB as your database and that the REPLACE function is available. It also assumes you have phpMyAdmin or a similar way to issue SQL (Structure Query Language) commands to the database. In phpMyAdmin, there is a SQL tab where you can type in and execute SQL. Just make sure you use a SQL tab for your database.

There are two tables that typically need fixing: phpbb_posts and phpbb_users. Steps:

  1. Disable the forum
  2. Backup the forum’s tables. Make sure it is a complete backup by downloading the extract, uncompressing if if necessary and looking at the end of the file. There should be SQL in there populating the phpbb_zebra table at the bottom of the file.
  3. Use phpMyAdmin or a similar tool to go into your database. If you are not sure which database you need to modify, look at your forum’s config.php file. The database name is in the file.
  4. You can examine the extent of the problem by first looking at each table. In these examples I assume your table prefix is phpbb_. The config.php file contains the actual table prefix, which may be different.
SELECT post_text FROM phpbb_posts WHERE post_text like '%IMG src="http://%';
SELECT user_avatar FROM phpbb_users WHERE user_avatar like '%http://%'
  1. To actually fix these, use the following SQL:
UPDATE phpbb_posts set post_text = replace(post_text, 'http://','https://') WHERE post_text like '%IMG src="http://%';
UPDATE phpbb_users set user_avatar = replace(user_avatar, 'http://', 'https://') WHERE user_avatar like '%http://%'
  1. Reenable the board
  2. You might need to purge the cache, but it should not be necessary.

After these steps, some users may notice that their avatar no longer serves and there is a big, ugly white space instead. They may try to change the URL in their Avatar settings back to http:// to restore it, in which case the problem may recur. This option can be disabled (see below). In general they should be encouraged to upload an avatar so it can be served from your web server, which will then serve it securely.

Preventing future insecure content

For avatars, the issue is due to allowing remote avatars. This can be changed: ACP > Board configuration > Avatar settings > Enable remote avatars > No

For posts, you can remove the permission to use the [img] BBCode. The easiest way to do this:

  1. ACP > Permissions > Group forum permissions > Registered users group > All forums
  2. For each forum, click on the Advanced permissions link, then the Content tab.
  3. Set Can use [img] BBCode tag permission to Never. Note: this will affect everyone, including special groups and administrators. If you want to have it affect only registered users, set it to No instead. Other groups however may retain the permission to post embedded images. You may want to use this pattern on other groups you have defined. 

Fixing blank spaces where embedded images should appear

Since blank space represent placeholders for external images that no longer exist, the URL may need to be corrected. You can try the MySQL Replace function above if you know the new pattern to use.

Alternatively, you can install the External Images as Links extension. This will substitute a clickable URL for the image. It’s likely the URL will lead to HTTP 404 error (not found), but it at least resolves the blank space image in the post.

 

Should I install phpBB?

What are you getting into when you install phpBB? phpBB, open-source forum software for the web, is often simple to install. Most web hosts have a scripting center that allows you to install it on a domain in a few clicks. But should you?

It’s not like there aren’t other forum solutions out there, although arguably phpBB is the one that has survived the longest. To name a few, there is commercial vBulletin software, myBB, Xenforo, Phorum and pUNbb. There are also forum plugins. For example, WordPress has BBPress and BuddyPress. Since I specialize in phpBB I can’t speak with much authority about other forum solutions. However, as a software engineer I can highlight what I think some of phpBB’s strengths and weaknesses are, the subject of today’s posts.

What is forum software exactly?

Before you decide on any forum solution, understand what forum software is. Forum software is not blog software. It’s not a place that you use to rant about stuff that interests you and which others can comment. It is software that allows lots of disparate people to discuss certain topic areas elegantly. It imposes discipline on the content it manages by keeping things organized in forums, topics and posts.

Forum software is used by discrete communities that have something in common and want to share that information in an open manner. Usually what they are discussing is pretty specialized. For example, it might be a support forum for a commercial or open-source product (phpBB.com uses phpBB for its support forum), or a fan site, a bunch of people who own a particular type of boat or plane, whatever! Forum software allows people to create and reply to topics. It’s designed to run independently of a framework. For example, the BBPress plugin for WordPress requires it to work as an add on to WordPress, which means that to use BBPress you must also be a WordPress user on the site. Similarly, Facebook groups can act a bit like a forum, but it requires you to join the Facebook enclave. Facebook however does not organize content in its groups into forums and topics. Most forum software is designed to be standalone, at this is certainly the case of phpBB. It’s not designed to work with WordPress or any other content management system. In our social media age, this is sometimes a drawback.

phpBB’s emergence

phpBB has a long and proud legacy. Version 1.0 was released in 2000, at just the moment that the PHP language became dominant on the web, replacing mostly a lot of Perl scripts. Timing was everything. It was written in PHP, used the popular free MySQL database and was free and open-source. “Open source” was kind of a new thing back then, but it was essential to its growth. Not only was it free, anyone could modify it.  So it got downloaded and installed like crazy. It’s still widely used today. Most support sites run on phpBB. This means you have probably used phpBB already, even if you aren’t aware of it. So it will seem comfortable and familiar, even if you don’t understand why.

Version 2 came a year later in 2001 and is still being used today by many sites because it is fast and lightweight. Version 3 was released in 2007, which thoroughly modernized it. Version 3.1 arrived belatedly in 2016. Its big feature was extensions, similar to WordPress plugins plus responsive styles, so things looked good on mobile devices. Prior to 3.1 if you wanted to extend phpBB’s functionality you installed “mods” that was code changes inside the source code, which made upgrading phpBB difficult. 2016 saw the release of version 3.2, the current version, which looks and behaves a lot like 3.1 but addressed some annoying issues mostly on the backend.

While phpBB was undoubtedly popular, updates were infrequent and its huge legacy base made it hard to push out new versions. Its team of core developers worked inefficiently together, in part because the tools for doing so were relatively primitive at the time. This allowed many other forum solutions to emerge to fill the feature gap while the phpBB group lumbered awkwardly forward into the future.

phpBB’s strengths

I first installed phpBB 2.0 in 2002 and have followed it since then. I have developed modifications and extensions, as well as generating good income from helping users upgrade and migrate their forums. In spite of the phpBB Group’s sometimes lumbering organization, it’s got some major strengths:

  • Institutional legacy. Simply because it’s been around so long, it tends to get widely installed and used. Those who have phpBB forums rarely move to other forum solutions.
  • Familiarity. Most likely you already know how to use phpBB because you have used it on various sites. While the forum/topic/post metaphor is hardly new, phpBB’s implementation of it garnered it a lot of attention and traction, so most forum solutions try to imitate it while addressing its perceived deficiencies.
  • A fanatical devotion to open source. The phpBB Group developers walk the walk on open source. They are really quite devoted to the whole idea of open source software, quite fanatical and arguably more than a little obsessed about it. They don’t give preference to any particular technology (except PHP and web standards like HTML, CSS and Javascript) and try to give you flexibility. For example, most forum solutions are written only for the MySQL database. Despite the fact that hardly anyone who has a phpBB forum uses databases other than MySQL, they support a whole host of other databases including Postgres, SQLite and Oracle.
  • Terrific support. phpBB’s support forums are phenomenal. You will likely find a dozen answers to your question with a simple search but if not a quick post will generate fast response, often from dozens of highly experienced support members, all volunteers. They are so good that in most cases the problems I encounter I don’t have to solve. I can find the solution on their support forums.
  • An anal obsession to standards. This is both a strength and a weakness. WordPress has now something like 40% of the web site market, but WordPress runs fast and loose. It’s not hard at all for people to create buggy plugins and non-optimal themes and WordPress will approve a lot of these. WordPress is a Wild West place where you are never quite sure if what you are adding on is crap or gold. That’s not a problem with phpBB. They go to extraordinary lengths to check their releases for bugs, running them against a host of security tools and making the base code pass thousands of detailed automated tests. I doubt there is an open source project that releases higher quality code. As an extension author, I am impressed and sometimes annoyed by how difficult it is to get my extensions approved. They inspect everything with incredible care and make sure you adhere to their voluminous and often somewhat obscure coding standards. This also makes things slow as there are plenty of extensions and styles in the review queue and reviews can take months. Rest assured though that officially approved extensions and styles are top quality.

phpBB’s weaknesses

  • Lack of agility. The phpBB Group’s tendency toward being anal also means they are not agile. It’s hard to bring out new versions of phpBB since everything must be nitpicked to death. Arguably this is also because there are tons of features and options in phpBB; look through all the Administration Control Panel’s various screens sometimes to get an idea of how many features can be changed, enabled and disabled. Its permissions system alone is awesomely powerful while awesomely obscure. When finally released, new versions tend to be very stable and rock solid but if you are an impatient person, your patience will definitely be tested and then some. On the other hand, their development practices are top notch. They use state-of-the-art testing, development and bug tracking tools. They have daily builds of their software to see what breaks.
  • Legacy architectureAdding new features tends to be excruciatingly difficult not because their code is not modular enough (this problem largely went away with phpBB 3.1) but because the database is so baked in. Many features would mean large changes to the database. Business logic is baked into many different programs, although phpBB 3.1 introduced classes (the whole /phpbb folder) that addressed a fair amount of this problem.
  • No multi-threaded topics. This means you can’t see a set of replies to a particular post within a topic, or get a hierarchical view of replies to a topic.
  • Standalone. It doesn’t integrate with anything, at least not elegantly. It won’t work seamlessly with your content management system, like WordPress. The closest it comes to this is that it supports authentication via LDAP (Lightweight Directory Access Protocol), but even so users must still create accounts on the forum to use it.

There is a lot more to this topic that I may delve into in future posts. But this post at least gives you a heads up. phpBB is great software: stable, reliable, well tested and industrial strength. If you can live with its functionality and limitations and are okay if the features change slowly at best, it’s still a terrific solution. If you need more agility from your forum solution, you might have to look elsewhere. However, any other solution you pick may not hang around. phpBB is eighteen years old and is likely to survive another eighteen years without a sweat.

 

You probably don’t want to host phpBB on Amazon EC2

Occasionally I do something new. This month something new meant helping to rehost a client on Amazon Web Services (AWS). AWS provides cloud computing services, and its EC2 service (EC = Elastic Computing) is probably its most popular service.

Cloud services provide scalable services. Also, you pay for what you use. They can be configured so that if there are spikes in demand the service will become “elastic”, scaling to meet demand.

I did not do this rehosting by myself. The client had another technical guy that set up and configured his AWS EC2 workspace. The forum is very large with 2.6M posts. In addition to rehosting the forum, I had to upgrade the client at the same time to the newest version of phpBB and move his WordPress site. This project literally took months to complete, although I was not working on it all the time.

I don’t know what Amazon Machine Instance (AMI) was set up in this case. I don’t think the AMI chosen was ideal because WordPress had technical issues that required fine tuning EC2 to get things to work. One thing I took away from the project though is that there is a “tax” if you move to AWS. The learning curve is steep and the tools available to you are miniscule.

Some lessons learned:

  • AWS doesn’t do any handholding. While you get a console to configure EC2, you don’t get any rich control panel like cPanel or Plesk to allow you to easily do complex things. So there is no File Manager, no phpMyAdmin (unless you want to install it yourself), no easy way to create mailboxes or send email. Instead, you need a UNIX geek. I was given a SSH key file and I had to use that to do my work. This meant doing pretty much everything from the command line.
  • Connecting to the AWS workspace was complex since I had to use SSH. The command lines used to connect with SSH were long to type in and easy to get wrong. I had to reference an argument to use the key (.pem) file I was given. When you work from the command line, it’s easy to type something incorrectly. So you often end up typing the command multiple times until you get it right. Closing and opening sessions becomes time consuming and a hassle.
  • To access the database, I had to do it from the command line. It took a while to get database credentials and since I had to do things from the command line again I had to type statements very carefully. I had to export and import databases but getting the syntax just right was challenging. I frequently had to go into the database to tweak things, which meant typing a lot of SQL statements. It’s a good thing I am very fluent in SQL. This really slowed me down. Had I had phpMyAdmin, I would have saved hours of time and hassle.
  • File permissions were a pain. The default user did not have the permissions to the web root folder, which meant becoming root and granting correct group and file permissions. Then secure FTP would work. These problems kept recurring which made the process quite tedious and time consuming.
  • The Apache web server was not configured correctly for WordPress. The AMI was apparently not tuned for WordPress, so it took research and carefully editing of a httpd.conf file to get the settings right. Then the web server had to be restarted from the command line, which is not intuitive, particularly since it had to be done as root.
  • Editing files became a pain. There was a lot of this, mostly tweaking forum styles, templates and configuration files. Since I had only the command line, I had to use a command line text editor. I chose nano, but it was still tedious. There is also a lot of command line navigation to get to the right folders where you needed to do stuff. This would have been easier if I had an editor on my computer that worked with SSH and .pem files. I had the former but not that latter, as I use an old copy of Dreamweaver to more easily edit files remotely.
  • I often had to become root to do things, like read the error_log file to troubleshoot issues.
  • The database import for the forum failed three times. I finally figured out the issue from the obscure error message. I had written two triggers for the client and that required CREATE TRIGGER privileges that were not granted to the default MySQL DBA. I had to snip these lines out of a 2GB+ export file to get the import to succeed. This has never been a problem on other hosts I have worked on.
  • Setting up HTTPS was a pain. It too required special permissions to create public and private key files, on the command line only, of course. It took many attempts before it worked and a valid certificate was installed. While we were waiting to test WordPress and the forum, it required using the long URL provided by AWS, which meant changing the configuration of WordPress and phpBB via the database.

I suspect that the wrong AMI was used or that using a better one would have prevented a lot of problems. In any event, the move to AWS turned out to be tricky, time consuming and a large hassle. For my client, it was an expensive endeavor. It turned out that all this work turned into my largest bill ever. In that sense it was good, although I would have been happier if it had spent a lot less time and a lot less of his money.

Now that my client is on AWS though, as long as it is tuned properly there will be some big advantages. Hosting may cost less in the long run, and the service should be more predictable and scalable.

If you take up a project like this, you will definitely need someone who has set up AWS EC2 instances successfully and will work through all these issues with you. Even so he could not do it alone. The migration took four days to complete, although the forum was functional in less than twenty four hours.

For very large forums that need scalability and high reliability, using a cloud service like AWS makes a lot of sense. However, a project like this should not be taken lightly. Most forum owners will probably be much more comfortable on a good shared host, or a good virtual or dedicated server. 

 

Avoid hosts owned by the Endurance International Group

As I have noted before, since I work with many clients I have developed hopefully informed opinions about many web hosts. Perhaps I should not paint with a broad brush but I do have one suggestion: avoid any web hosts owned by the Endurance International Group.

Web hosting tends to be a low profit business. With so much competition, customers will shop around for the best deal. This results in many hosts offering cheap plans for $5 or $10 per month. Web hosts can hope to find profitability in volume but since there is a lot of competition profitability usually comes from consolidation. The Endurance International Group buys web hosts. It looks like they find profitability through throwing all these companies under one umbrella and one hosting center.

I used to host with Hostgator. I found their support good and their infrastructure above average, yet their pricing was very reasonable. Then they were bought out by the Endurance International Group. Almost immediately afterward their support became crappy and I noticed delays accessing my domain as well as infrastructure related issues. When my hosting contract was over, I was happy to move somewhere else.

When you call these companies for support, you are immediately placed into a third-level support queue. After you finally connect with a human, these brain-dead support people follow scripts that are designed basically to not solve your problem and make you go away. Moreover, I found myself far more knowledgable about hosting and how to solve problems than they were. They could rarely even cover the basics. If you needed real help I found I had to badger for second-tier support.

All this is to keep their costs low since one support center for dozens of companies is obviously cheaper. But it results in inferior service, as evidenced by my experience with hostgator.com.

Endurance International Group own a lot of hosts, most pretty obscure. Among those they purchased you may be familiar with include Hostgator, Bluehost, Hostcentric, iPage and Site5. You can see a full list of the brands they bought on this Wikipedia page.

Given the low margins, the hosting business is likely to continue consolidating. There is certainly a lot of smoke and mirrors in this business. What used to be good hosting can turn into poor hosting pretty quickly when they get acquired. This is true of MediaTemple, at least it’s Grid Service, based on my latest experience since it was bought by GoDaddy. 

My current hosting recommendations, last updated April 2019

Right now my recommended hosts include Siteground for most hosting and Rackspace for dedicated and virtual server hosting.

This site is hosted on Siteground. I have a number of reasons why I prefer Siteground:

  • Their infrastructure is completely solid state, end to end. This means the latency in getting data off your website is minimal.
  • In their cPanel you can set up https very easily using free Let’s Encrypt security certificates. They automatically handle the renewal of these certificates when they expire every 90 days.
  • Their support is quick and excellent. I rarely wait more than a minute for technical support.
  • They are not owned by Endurance International Group, or any conglomerate.

If you choose to host or rehost with Siteground, please use my affiliate link. This way I earn a small commission. You will not pay extra.

If you need help moving your website to Siteground, or a new host, I can certainly help. You might want to send me a service inquiry. If you choose Siteground, you should be able to have them move your site for free, but you have to ask.

It’s quite clear to me though that you are likely to be unhappy with any hosting owned by the Endurance International Group. So avoid.

Why you might want to use SMTP to make emailing more reliable

Emailing from phpBB is often problematic. In a previous post I looked at various ways to make sending email from phpBB more reliable.

In this post I look at why you might want to configure phpBB to use SMTP (Simple Mail Transfer Protocol) to send emails. If you are using a Windows web server (IIS), this is usually required. You can change these settings in ACP > General > Client communications > Email settings, which can be set halfway down the page.

By default, phpBB hands off email to PHP using PHP’s mail function and hopes for the best. If the mail function returns FALSE, the email should not have gone out. You won’t necessarily be know if it fails, however. Known email sending failures appear in phpBB’s error log: ACP > Maintenance > Error log.

Even if the PHP mail function returns TRUE, it may be a false report. In many cases TRUE only means that the mail was accepted. Whether TRUE means “accepted” or “successfully sent” depends on how your host’s email server is configured.

Once accepted by the email server, the email is likely to be closely examined. If it has some of the markers of being spam, it will probably get blocked from actually being sent out. In this case, you probably won’t know about it. You can greatly improve the odds that emails will go out successfully if you:

  • Create an email account that uses your forum’s domain, ex: admin@mydomain.com
  • Assert this email address in phpBB: ACP > General > Client communications > Email settings. Set this email address for Contact email address and From email address.

By default on Linux web servers (most typical kind of hosting), PHP’s mail function sends outgoing email to a sendmail process. Essentially, emailing is the job of your server’s operating system, so you can’t control it. Because you can’t control it, sendmail will be configured generically. Any other domains on the server you are using will send email out through the same sendmail process.

However, if you use SMTP to send email instead, you have to authenticate yourself with the SMTP mail server. By properly authenticating yourself, emails are likelier to go out. The downside is that it’s a bit of a hassle to set up SMTP. I discovered this on this site when I moved my hosting to siteground.com. It uses WordPress which by default also uses sendmail. Emails weren’t going out. A call to Siteground’s tech support revealed that SMTP was a better way to go. They provided the credentials to use and now my WordPress email notifications go out quickly and reliably. This should work for phpBB too.

Don’t assume that you can use an external SMTP server like GMail. Many web hosts will block outgoing SMTP email. If it’s allowed, by all means go ahead. You should check with your web host for the proper email settings to use.

One possible downside is that outgoing email quotas are likely, but that’s probably also true if your host uses sendmail. Your web host can tell you what policies if any apply to your hosting. See the previous post for instructions on how to properly set your email package size and to set up a system cron, if necessary. Make sure it works by sending a test email, an option available on the email settings page.

Cleantalk extension for phpBB can remove spam posts, plus its spam firewall feature is very useful

This is an update on an earlier post on removing spam posts.

Removing spam posts is hard because it requires actually reading the post and deciding if the post is spam or not and then using moderator tools to remove these posts. If your forum is overwhelmed with spam posts, this is a Herculean endeavor. Ideally though posts could be “read” by software and it would make the judgment on whether it is spam or not.

The Cleantalk extension for phpBB 3.1.x and 3.2.x can do just this as well as lots of other really cool tricks. My customers love Cleantalk, but the service is not free. However, it is so inexpensive that it easily justifies spending $8/year for the service. You can subscribe on the Cleantalk website. As of this writing, you can try it for free for 7 days. After 7 days, it won’t bring down your forum but it will stop working.

What is Cleantalk?

Cleantalk is essentially a huge database of addresses of known spammer sites. While it’s not perfect, based on the experience of my clients it is about 99% perfect. I originally recommended it as a spam registration solution for my clients. It still does that but is less necessary since phpBB 3.2. This is because since phpBB 3.2, version 2 of Google’s reCaptcha is supported. Unless it gets hacked, as long as you have it properly configured as a spambot countermeasure it should prevent virtually all spam registrations.

However, it has two powerful features that still keep it relevant for phpBB forums.

Cleantalk ACP Interface
Cleantalk ACP Interface

Installing and enabling Cleantalk

Cleantalk is installed like any other extension. While it can be downloaded from phpbb.com, you should download it from Cleantalk instead or from its GitHub page. This is because as of this writing the version on phpbb.com does not include the spam firewall feature, and you will probably want to enable this feature. You can access it through the Administration Control Panel: ACP > Extensions > Antispam by Cleantalk. Before you can do much with it you have to enter your Cleantalk key which you can get from their website or by pressing the button in the extension that should retrieve it for you.

Removing spam users and spam posts

As you can see from the image, once the extension is enabled and the key is properly configured there is a prominent Check users for spam button on its page within the Administration Control Panel. If you have lots of users, it may hang. Based on my experience though the next time you go into its interface you will see a list of potential spammers.

As I said, it is not perfect. So I recommend that for users with posts to check these out these users topics to make sure their posts are spam before deleting them. For those you want to delete, check the boxes next to their usernames and then press Delete marked. You can also press Delete all to remove all users and their posts. You may have to go through many pages to delete all spam users and their posts, but this is obviously much faster than doing a visual inspection of all your posts.

Spam firewall

This is a new feature which as of this writing is not available if you download the extension from phpbb.com. It keeps almost all spammers from hitting your site at all. Instead, Cleantalk’s servers grab it first. In the event the user is legitimate, there is a link that will take them to your website.

Why is this useful? Because it reduces the stress on your server by limiting it to legitimate traffic only. It speeds up the performance of your forum and makes it less likely that you will have to pay for the cost of a higher class of hosting to handle your traffic. Isn’t that worth $8 a year?

Stopping contact form spam

Cleantalk has one other useful feature: the ability to stop contact form spam. Of course you can disable the contact form (ACP > General > Contact page settings) and that will solve that issue. Or you can have Cleantalk essentially moderate it for you, passing on only valid contact forms to you. Simply check that option on the extension’s page and submit the form. Somewhat oddly, the phpBB group did not tie the contact form to the spambot countermeasure feature of phpBB. Perhaps that will come in a future release.

In any event for forums that get lots of spam and/or lots of traffic, using the Cleantalk service with the Cleantalk extension for phpBB is a no-brainer providing you know about it. Now you do!