Forum Shoutbox

[Kybyrian]

Until we can get a proper announcement on the current state of the shoutbox/chatbox, I'm going to plant this thread here in light of what the current situation is.

The shoutbox is currently deactivated, and will likely remain inactive until well after the holidays.

"Why exactly is it disabled?" you may ask. Well, allow me to clear up some of the speculation.

- The shoutbox is not disabled because of the current server problems we've been having

- The server problems actually did not even start until well after the shoutbox was removed​

- The shoutbox will not remain gone permanently

-Mases recognizes the importance of the shoutbox to the community, and knows that maintaining it is the best course of action​

Now, let's get to the real reason the shoutbox is not currently here.

The shoutbox is currently deactivated in preparation for the holidays

Though the shoutbox has not been causing any current database issues, it still uses a lot of CPU (resources) dedicated to the server. With the projected influx of visitors over the holidays, this could present a problem for the site overall. Once these current issues are fixed, we'll be ready for the holidays. However, we don't want a lack of resources to handle the amount of visitors that will be flocking to the site during this time period, so we took a precautionary measure by disabling the shoutbox to ensure we don't have any grossly extended downtime. The shoutbox could possibly result in server issues if it were active during times of especially high activity. Currently, we are not experiencing this type of activity, but the shoutbox is disabled in preparation for it. The shoutbox is disabled because of possible future issues, not current ones.

The staff is sorry for the convenience this may have caused and the lack of communication.

I hope this helps everyone out.

Thanks,
Kybyrian

 

[Emma]

The shoutbox is not capable of using the CPU resources you claim it does. The software just doesn't work like that. And you contradicted yourself. You said it wasn't because of server issues, then went right and said it was because of server issues. You are lying to us. Whatever the real reason for removing it is, it has nothing to do with technical problems and it's entirely because of some personal preference. One viewer loading any page causes more server load than ten people talking on the shoutbox. It simply isn't capable of the kind of load you claim it is. Instead of lying to us to cover up the fact that you simply don't know, just tell us you don't know what the problems are and stop removing features based on reasons you cannot support with any proof.

 

[Kybyrian]

You said it wasn't because of server issues, then went right and said it was because of server issues. You are lying to us.

No, you are twisting what I said.

- The shoutbox is not disabled because of the current server problems we've been having

The shoutbox could possibly result in server issues if it were active during times of especially high activity.

These are two completely different situations and two completely different statements. I said the shoutbox wasn't removed because of current server issues. Then I said the shoutbox was removed because of possible future server issues and that we only want to prevent that.

And yes, the shoutbox is capable of using the resources I claim. You don't know anything about it, I'm afraid. I observed it myself while I was an administrator. Is it some underlying issue that's causing our CPU usage to skyrocket while the shoutbox is enabled? I don't know, and we don't have the resources or people to figure that out. The only thing that's certain is that CPU usage is significantly higher when the shoutbox is enabled. You have not seen the numbers. Our current situation is not related to CPU usage, however. This is only precautionary for when we need the extra CPU.

One viewer loading any page causes more server load than ten people talking on the shoutbox.

Your analogy is entirely incorrect. It's not even about how many people are talking on the shoutbox or when anyone is talking on the shoutbox. I can't explain to you why the shoutbox has a high CPU load, and you can't explain to me why it doesn't because neither of us actually know anything about it. The difference is that I won't pretend I will. The only thing I do know is that CPU usage is higher while the shoutbox is enabled.

I only want to help out, and I'm not forum staff myself anymore either. I only work close with them. I was confused and did not know for a long time why the shoutbox was gone either. I have no reason to lie to you and no reason to want the shoutbox to be gone.

 

[(◕‿◕✿)]

First off,

- The shoutbox is not disabled because of the current server problems we've been having
- The server problems actually did not even start until well after the shoutbox was removed

and

The shoutbox could possibly result in server issues if it were active during times of especially high activity.

kind of contradict each other. More to the point, though, these claims don't really add up. Forum software is heavily dependent on SQL queries to accomplish things, with the interspersed JavaScript/AJAX, of course. Every time someone does anything on the forums, at least one query is executed. This means that the forum software (PHP) communicates with the database (MySQL in this case, though it can sometimes be other SQL implementations depending on the server and scale). This takes CPU time, of course, and puts tiny amounts of strain on the server.

Let's take the example of using the New Posts function. vBulletin has a set of functions defined that form an SQL query based on input from users and prepare it for execution, which retrieves data from the database. This is more than one query alone, per user, per search. You often have many people doing this at once. Or perhaps a more pertinent example: posting in itself. This executes a combination of queries that updates many, many fields in several tables on the database. users has to be updated, posts and threads, as well as many other tables scattered across the vB database. Again, you often have many people executing this at the same time, resulting in exponentially increasing numbers of queries.

The main source of resource use with respect to the shoutbox is the fact that it uses AJAX (Asynchronous JavaScript and XML) to update the frame in near real-time. This means that every 2 or 3 seconds, the shoutbox is refreshing, looking for new shouts. This is similar to people refreshing the index page periodically (or lots of users requesting similar pages every so often). There's regularly over 100 guests during the day (North American time) and often many at night, who are constantly requesting pages, many of them executing far more queries than the shoutbox alone would.

So, the argument that the shoutbox alone eats away at the CPU like you claim just doesn't hold up. There's dozens of other features on both the main site and the forums in particular that use up lots of resources. People are constantly utilizing features that are similar in intensity to the shoutbox, even guests on the forum. Lastly, during the time we had the Inferno Shoutbox, the forums were actually a bit more active, and lots of people constantly used that shoutbox, yet there were no complaints of high CPU usage.

 

[Kybyrian]

Lastly, during the time we had the Inferno Shoutbox, the forums were actually a bit more active, and lots of people constantly used that shoutbox, yet there were no complaints of high CPU usage.

The reason we removed the inferno shoutbox was because of high CPU usage and server errors.

Let me try to provide some technical insight into it now.

Do I know why the shoutbox is creating high CPU usage? No.
Is it? Yes.

But, let's try to see why it might. I honestly have nothing to do with the shoutbox being gone, and I can only relay what I hear from the rest of the staff, including Mases.

It's possible that we could solve this problem rather easily. Here we maintain archives. In fact, we maintain a lot of archives. How large have they gotten? I don't know, and I can't check anymore because I'm not an administrator.

If these archives are, say, half a large as the post database (which they very well could be), then that means every single person who visits the forum index is loading that entire chat database every single time they visit. That would put a very large strain on the server. We could solve this by deleting out archives. I don't know how often our archives automatically clear, but with our activity we could be building very large archives all the time. We view the archives as a simple necessity and will not remove them. As I recall, archives were part of the problem with the inferno shoutbox as well. I'll mention this to Locke and see if we can't get the CPU issues sorted out.

Also, I guess I should mention this as well. Not only could we be experiencing abnormal CPU usage for this reason, but take a look at what you said:

People are constantly utilizing features that are similar in intensity to the shoutbox, even guests on the forum.

By this logic, wouldn't CPU usage double because of the presence of the shoutbox?

There's regularly over 100 guests during the day (North American time) and often many at night, who are constantly requesting pages, many of them executing far more queries than the shoutbox alone would.

I don't doubt this, but many of the guests that show in the active users list load less than ~10 threads overall. This listing also shows people last active in the last 30 minutes, meaning many of the guests you see aren't even browsing anymore, and most don't browse for any more than a few minutes. You also equated shoutbox refreshing to refreshing the forum index, but not every user will reload the forum index every 2-3 seconds, or even every minute.

Though normal queries will indeed use much more CPU than the shoutbox, the shoutbox is still a healthy contender. I, and no one else, never said that the shoutbox was causing CPU usage to skyrocket more than the forum itself. I only said it was significantly contributing to high CPU usage. If what you said it right, then in combination with archives we could easily use a lot of resources maintaining the shoutbox. The shoutbox itself, according only to what you said and your comparisons, also uses a healthy percentage of the CPU that the forum itself uses. Either way you look at this, the shoutbox is using up quite a bit of CPU, and maybe we can cut down on that with the issue I mentioned earlier.

 

[Kybyrian]

Refreshing every 2-3 seconds certainly does increase the strain, but it's not an amazingly huge deal.

As for the archives, you set automatic archive clearing separately from how many shouts people can see in the archives. Even if you could only see 100 shouts in the archive, it could be set to never prune itself. That would mean that you could accumulate an archive that easily stores over 300,000 shouts while your users can only view several hundred of them.

We don't have any true techies to determine the exact nature of the problem and how it's related to the shoutbox, unfortunately. The best foreseeable solution is to disable the shoutbox, but I can assure you that it will be back. It wasn't disabled as the cause of the issues we're having right now, but because it could cause problems over the holidays as the userbase gets larger. The best solution would be to bring on board someone who can sort out the shoutbox CPU usage problem, however. I don't know if it's inherent or related to a different problem or not. None of us do.

 

[Clank]

Thank you for informing us about the situation and clearing up the misconception about these issues.

And I do know the kinds of loads that shoutboxes put on servers due to first-hand experience, and I appreciate you backing me up on what I have been saying all along. Thank you, it means a lot to me.