This website forum sucks

[woody1]

And don't get us started on the quality of the moderating!

There won't be that much to moderate the way things are going.

[DoctorStrangelove]

Seems to be sucking particularly hard for some this morning. .

This posted despite the 500 error.

[sadkingbilly]

yeh, - posts disappear for no apparent reason.
except maybe the BibleThumpingTaffies

[Uncle Albert]

And don't get us started on the quality of the moderating!

Or the quality of the posting! I doubt that any passing Cornish farmers would bother popping in anymore.

[woody1]

Although "Post New Thread" seems to be totally buggered today, what's a bit weird is that most of the time the problems seem to be intermittent.

I don't know about "Accounting/Legal" but "Technical" doesn't seem to be affected by gremlins as much as "General".

[NickFitz]

The fundamental problem is that a good few years ago now, vBulletin's developers learned about "Object Oriented Programming" along with the concept of "Object-Relational Mapping". Enthused by this new knowledge, they embarked on a ground-up rewrite of the perfectly adequate vBulletin forum software.

Now, there are various kinds of thing that can be considered as "objects" in forum software: a thread, a post, a subforum, a private message, a user profile, and so on. Put various instances of these things together, and you get a forum. And if you're in the early days of understanding the benefits of abstraction, you get carried away, and decide that all these things that have the property of being "puttable together" must all, in some deep and meaningful way, be of the same kind. So you create a superclass, which all these things' classes will descend from, and you call this abstract thing that everything is a kind of, a "Node".

And because you don't realise that implementing an Object-Relational Mapping does not, in fact, imply a one-to-one correspondence between classes in the software and tables in the database, you take all those things that used to be in separate database tables - one for subforums, and one for private messages, and one for threads, and one for posts, and others for all the other things - and you put them all in one table, which contains all the Nodes, because they're all a bit different from each other but in a way, you feel they're all the same.

While you are doing this, the business is sold, and the new owners decide that the way to make the product stand out in the market is to add a load of other features (articles, and blogs, and events, and various other things) and guess what? They're all Nodes too!

But then the new owners decide this rewrite is taking too long, and experienced developers are too expensive. So they get rid of the experienced developers, and they tell the ones that are left that time's up, so "Ship it!"

It turns out that many of the forums that use the software are quite small, without many users or many threads or many posts, and they also quite like being able to write articles and start blogs, so a lot of people carry on using the software, even though it's fundamentally flawed by the terrible decision that everything is a Node.

But along comes CUK, with lots of threads, and lots of posts, and lots of private messages.

And when a CUK user goes to look at the forum home page to see what people are talking about, or goes to a thread to see what they're saying about it, the forum software goes to the database and asks for the relevant Nodes.

And the database's Nodes table has millions of rows in it: a row for every subforum, and a row every thread, and one for every post, and every PM, and loads of other things, all in one table.

And the database has to load the entire contents of the table into memory, and hunt through it to find the ten threads to show on the forum page, or the ten posts to show on the thread page. Or if you go to look at your PMs, it has to load the millions of rows into memory to search through them all for the PMs that are addressed to you.

And it has to do this for everything: millions of rows, with the database constantly loading them from disk and scanning back and forth to find the stuff it needs to show you a page.

There's cacheing to make things a bit quicker. But that doesn't help much: every time somebody posts, the cache for that thread gets invalidated and guess what? Yes, millions of rows to search again!

And that's why the forum software sucks

[jamesbrown]

Was this rewrite PM'd by suity?

[NickFitz]

Was this rewrite PM'd by suity?

The experienced developers who left vBulletin set up their own forum software company, Xenforo. However, vBulletin then sued them and they were tangled up in lawsuits for over a decade, during which time there was the ever-present risk that they'd lose and the product would vanish.

However, in the last year or two, the lawsuits were settled. So it seems to me that it's a viable substitute. When I have time I'm going to have a proper look at it and, if it seems good, I'll see if the powers that be at CUK are interested in switching. I know it's in use by some very big sites - I believe Wired use it for their forums.

[quackhandle]

So they denormalized a perfectly working database?

Great job.

qh

[dsc]

The fundamental problem is that a good few years ago now, vBulletin's developers learned about "Object Oriented Programming" along with the concept of "Object-Relational Mapping". Enthused by this new knowledge, they embarked on a ground-up rewrite of the perfectly adequate vBulletin forum software.

Now, there are various kinds of thing that can be considered as "objects" in forum software: a thread, a post, a subforum, a private message, a user profile, and so on. Put various instances of these things together, and you get a forum. And if you're in the early days of understanding the benefits of abstraction, you get carried away, and decide that all these things that have the property of being "puttable together" must all, in some deep and meaningful way, be of the same kind. So you create a superclass, which all these things' classes will descend from, and you call this abstract thing that everything is a kind of, a "Node".

And because you don't realise that implementing an Object-Relational Mapping does not, in fact, imply a one-to-one correspondence between classes in the software and tables in the database, you take all those things that used to be in separate database tables - one for subforums, and one for private messages, and one for threads, and one for posts, and others for all the other things - and you put them all in one table, which contains all the Nodes, because they're all a bit different from each other but in a way, you feel they're all the same.

While you are doing this, the business is sold, and the new owners decide that the way to make the product stand out in the market is to add a load of other features (articles, and blogs, and events, and various other things) and guess what? They're all Nodes too!

But then the new owners decide this rewrite is taking too long, and experienced developers are too expensive. So they get rid of the experienced developers, and they tell the ones that are left that time's up, so "Ship it!"

It turns out that many of the forums that use the software are quite small, without many users or many threads or many posts, and they also quite like being able to write articles and start blogs, so a lot of people carry on using the software, even though it's fundamentally flawed by the terrible decision that everything is a Node.

But along comes CUK, with lots of threads, and lots of posts, and lots of private messages.

And when a CUK user goes to look at the forum home page to see what people are talking about, or goes to a thread to see what they're saying about it, the forum software goes to the database and asks for the relevant Nodes.

And the database's Nodes table has millions of rows in it: a row for every subforum, and a row every thread, and one for every post, and every PM, and loads of other things, all in one table.

And the database has to load the entire contents of the table into memory, and hunt through it to find the ten threads to show on the forum page, or the ten posts to show on the thread page. Or if you go to look at your PMs, it has to load the millions of rows into memory to search through them all for the PMs that are addressed to you.

And it has to do this for everything: millions of rows, with the database constantly loading them from disk and scanning back and forth to find the stuff it needs to show you a page.

There's cacheing to make things a bit quicker. But that doesn't help much: every time somebody posts, the cache for that thread gets invalidated and guess what? Yes, millions of rows to search again!

And that's why the forum software sucks

That's why I always say "feck classes" or actually I should say "feck turning everything to a class".