Mahz

Mahz's Dev Journal

Hey, I'm back. Catching up with some Guild duties.

When I'm away, I still visit the Guild and check the server stats to ensure it's not down. It just seems that switching between guild-mode and no-guild-mode is my proven method for avoiding burnout over the last decade.

One thing I need to do is award the contest winners.

Bot Reporting 2.0

@Duthguy I snipped your link out so that another mod doesn't accidentally nuke it.

You linked a legit roleplay between our few Korean users. One of them was accidentally nuked before, too. Not spam. :(

Mahz's Dev Journal

I found the pattern (tho there may be more) that the BBCode parser seems to hate. Before I post it, I want to figure out a patch.

I asked via PM the last person that caused the 100% CPU issue to edit their BBCode to remove the pattern from the post they were trying to make, and it worked without bringing down the server. No downtime since they made the change.

I'm glad I finally found the issue, but I hope I figure out a patch soon.

UPDATE: Pushed up a weak hotfix. Will keep digging.

I'll keep an eye on the downtime. If the 100% CPU issue is triggered again, I'll know how to find the post that caused it, and I can keep doing this until I've patched the most common causes.

Mahz's Dev Journal

Well, since it's an issue with the BBCode parser, any time someone tries to post/edit a post with the offending BBCode pattern that drives the parser insane, it's going to bring that server back down again.

Mahz's Dev Journal

snip

Mahz's Dev Journal

Think I found the issue.

The BBCode parser takes exponential time on some posts. I've been suspecting that the 100% CPU issue was caused by my major platform upgrade a month or two ago, and this would make sense since the underlying regular expression engine was probably updated in some possibly minor way that is now susceptible to a near-infinite loop given the same parsing logic we've been using for over a year.

I'll try to narrow it down further so that I can patch the offending logic of the BBCode parser.

Mahz's Dev Journal

Oh wow, wtf? Been visiting Texas so forgot to do my daily checkup on the Guild. Looks like the 100% CPU issue lives. At least I now know it's not a database issue.

Rebooted the servers. I'll look for a more automated way to reboot the servers.

Mahz's Dev Journal

@Buddha Thanks, that's awful: github.com/danneu/guild/issues/146

A cybernetic Swami has wondered in from the wastes...

Welcome back

Bot Reporting 2.0

@ClocktowerEchos For the same reason one might wash graffiti off their wall.