Show Posts
46
News and Events / Welcome to the new forums / explanation for downtime
« on: February 08, 2013, 09:09:14 PM »
Greetings Carcassonne fans!
Over the past six to eight months we've had a LOT of problems with our forums. On behalf of the management, I would like to express our appreciation for those who waited patiently and/or suffered through the whole ordeal. During the downtime, the amount of complaining slowly increased. I'd like to remind everyone that this site is run by volunteers in our spare time. This game is very important to us, and we don't take lightly the fact that we are the semi-official English language community for Carcassonne. I want to offer, in one place, as complete of an explanation as possible for why the downtime was so long, and what was happening during that time.
The problems began towards the beginning of Summer 2012 when people started to have problems after making a post. Instead of redirecting back to the thread, they were sent to a blank page. The problem was investigated but the cause or resolution could not be found. We slogged through it and continued posting. A few months later, the problems got worse. People were having trouble logging in and couldn't post at all. Another round of investigation, but the cause and resolution still eluded us.
Eventually we decided to try reinstalling the forum software, but kept running into problems. It took a VERY long time trying to find people that were able to help solve the problems, and each time a problem was solved, another problem surfaced. Then we tried upgrading to the new version of the forum software, and that's when disaster struck. The database got corrupted, and our backups wouldn't restore.
During this time, the forums were completely inaccessible. Gantry escalated the issue to our web hosting provider, and after a LOT of investigation it was discovered that back when the problems began, somebody had hacked the database and inserted a bunch of garbage. This garbage prevented a successful complete backup, which is why we couldn't restore from the recent backups. The last successful backup was restored, and the forums were sent back in time, but the posting problems continued and got worse again. (I was not able to log in and had to relay messages to Kettlefish who would post them on the forum for me.)
As the level of complaining on BoardGameGeek increased, so did my stress level. There was nothing I could do to fix the problem, since I didn't have as much access to the web server as Gantry, and Gantry was constantly away on a business trip or battling some form of illness. When he had the opportunity, Gantry was trying everything he could think of fix the forums. It was bad enough losing the small amount of posts from the past six months, and we didn't want to lose any more. A lot of people worked very hard building this content and we value it highly.
During the downtime, I created the official CarcassonneCentral Facebook page. (If you haven't "liked" it yet, please do so now!) This enabled me to continue collating Carcassonne-related news from various sources: Z-Man Games, Hans-im-Gluck, BoardGameGeek, and the Carcassonne iPhone App. Also during the downtime, many things were happening in Germany. A new small expansion was released, the Winter Edition and the first winter expansion were released, a fan gathering was held with Klaus-Jurgen Wrede in attendance, a new German language forum was created, and some guy in Hungary started manufacturing CarcassonneCentral expansions and accessories and selling them on eBay. (Somehow I ended up squarely in the middle of a giant legal battle where I represented the expansion authors, myself as one of the expansion authors, CarcassonneCentral, and Hans-im-Gluck all at the same time. I got a very nice e-mail from Moritz Brunnhofer afterwards thanking me for it and apologizing for not getting their own legal team involved.)
This seems like an appropriate point to acknowledge our community member Kettlefish (aka Marion). She spent a lot of time on the phone with various people at Hans-im-Gluck, getting answers and clearing up various issues, and also travelled by train to meet with them in person at Spiel 2012 where she grilled them with questions for the Complete Annotated Rules. She also travelled to the fan gathering and met with Klaus-Jurgen Wrede to talk about what is next for Carcassonne. She spent a lot of time working with Chris Ober on the Completed Annotated Rules, and I would be remiss if I didn't acknowledge how much work he did during the downtime also. I think he published two versions plus a new Winter Edition CAR. The CAR is now fully up-to-date with all the German expansion releases.
Getting back to what was going on here in Canada... seeing that our efforts to fix the forums were not getting any closer to success, it was time to think outside the box. One particular person on BoardGameGeek, who shall remain nameless, was threatening to start his own Carcassonne forum. Word came down from Hans-im-Gluck that they didn't want another English-language community, they wanted CarcassonneCentral. I suggested to Gantry that it was time to give up on trying to fix the forums and create new ones, but leave the old forums running in read-only mode so that nothing would be lost. This wasn't the ideal solution, but it was the only way to get away from the database corruption issue. That was just a few days ago, and here we are today.
Gantry spent the last three evenings setting up all the forums, membergroups, and permissions. This evening I had a nice chat with Bixby on the phone and he gave me some recommendations for add-ons to install, which I've done. The most difficult work is finally done, but now the most time-consuming work begins. We need to reload all the downloads, fill the forums with threads, and bring the community back together.
Over the past six to eight months we've had a LOT of problems with our forums. On behalf of the management, I would like to express our appreciation for those who waited patiently and/or suffered through the whole ordeal. During the downtime, the amount of complaining slowly increased. I'd like to remind everyone that this site is run by volunteers in our spare time. This game is very important to us, and we don't take lightly the fact that we are the semi-official English language community for Carcassonne. I want to offer, in one place, as complete of an explanation as possible for why the downtime was so long, and what was happening during that time.
The problems began towards the beginning of Summer 2012 when people started to have problems after making a post. Instead of redirecting back to the thread, they were sent to a blank page. The problem was investigated but the cause or resolution could not be found. We slogged through it and continued posting. A few months later, the problems got worse. People were having trouble logging in and couldn't post at all. Another round of investigation, but the cause and resolution still eluded us.
Eventually we decided to try reinstalling the forum software, but kept running into problems. It took a VERY long time trying to find people that were able to help solve the problems, and each time a problem was solved, another problem surfaced. Then we tried upgrading to the new version of the forum software, and that's when disaster struck. The database got corrupted, and our backups wouldn't restore.
During this time, the forums were completely inaccessible. Gantry escalated the issue to our web hosting provider, and after a LOT of investigation it was discovered that back when the problems began, somebody had hacked the database and inserted a bunch of garbage. This garbage prevented a successful complete backup, which is why we couldn't restore from the recent backups. The last successful backup was restored, and the forums were sent back in time, but the posting problems continued and got worse again. (I was not able to log in and had to relay messages to Kettlefish who would post them on the forum for me.)
As the level of complaining on BoardGameGeek increased, so did my stress level. There was nothing I could do to fix the problem, since I didn't have as much access to the web server as Gantry, and Gantry was constantly away on a business trip or battling some form of illness. When he had the opportunity, Gantry was trying everything he could think of fix the forums. It was bad enough losing the small amount of posts from the past six months, and we didn't want to lose any more. A lot of people worked very hard building this content and we value it highly.
During the downtime, I created the official CarcassonneCentral Facebook page. (If you haven't "liked" it yet, please do so now!) This enabled me to continue collating Carcassonne-related news from various sources: Z-Man Games, Hans-im-Gluck, BoardGameGeek, and the Carcassonne iPhone App. Also during the downtime, many things were happening in Germany. A new small expansion was released, the Winter Edition and the first winter expansion were released, a fan gathering was held with Klaus-Jurgen Wrede in attendance, a new German language forum was created, and some guy in Hungary started manufacturing CarcassonneCentral expansions and accessories and selling them on eBay. (Somehow I ended up squarely in the middle of a giant legal battle where I represented the expansion authors, myself as one of the expansion authors, CarcassonneCentral, and Hans-im-Gluck all at the same time. I got a very nice e-mail from Moritz Brunnhofer afterwards thanking me for it and apologizing for not getting their own legal team involved.)
This seems like an appropriate point to acknowledge our community member Kettlefish (aka Marion). She spent a lot of time on the phone with various people at Hans-im-Gluck, getting answers and clearing up various issues, and also travelled by train to meet with them in person at Spiel 2012 where she grilled them with questions for the Complete Annotated Rules. She also travelled to the fan gathering and met with Klaus-Jurgen Wrede to talk about what is next for Carcassonne. She spent a lot of time working with Chris Ober on the Completed Annotated Rules, and I would be remiss if I didn't acknowledge how much work he did during the downtime also. I think he published two versions plus a new Winter Edition CAR. The CAR is now fully up-to-date with all the German expansion releases.
Getting back to what was going on here in Canada... seeing that our efforts to fix the forums were not getting any closer to success, it was time to think outside the box. One particular person on BoardGameGeek, who shall remain nameless, was threatening to start his own Carcassonne forum. Word came down from Hans-im-Gluck that they didn't want another English-language community, they wanted CarcassonneCentral. I suggested to Gantry that it was time to give up on trying to fix the forums and create new ones, but leave the old forums running in read-only mode so that nothing would be lost. This wasn't the ideal solution, but it was the only way to get away from the database corruption issue. That was just a few days ago, and here we are today.
Gantry spent the last three evenings setting up all the forums, membergroups, and permissions. This evening I had a nice chat with Bixby on the phone and he gave me some recommendations for add-ons to install, which I've done. The most difficult work is finally done, but now the most time-consuming work begins. We need to reload all the downloads, fill the forums with threads, and bring the community back together.