Solver:	CivIV has been developed with an unprecedented level of fan input. Tell us a bit about the play session. What were the main ways it helped to develop CivIV, and how was your experience with it?
Sirian:	The play session was all about feedback. No concept went unchallenged. No element survived without being questioned, poked, and thoroughly examined. Fall 2001, screenshot from my second-ever game of CivIII. I observed stuff there that Soren has been hearing about for years. Lots of fun, but also lots of instances where the AI could have performed a lot better! If diplomacy is the overarching issue of single player, this game was a clinic on everything about CivIII diplomacy that could stand to have been improved! I am not kidding. This was also a huge map "archipelago". You see any chains of small islands? I don't! Where are the islands? (They'll be there in CivIV.) There were three main components to the session: single player, multiplayer, and mods. Jesse and Soren hand-picked a CivIII community leader, a fan, to administrate and lead each area. I worked very closely with my counterparts, Jesse Fletcher (Friedrich Psitalon) and Martin Isaksen (Isak). The first [thing] that they had us do when we arrived was to get to know one another and gain some respect for one another's expertise. Soren would say, "Civ is too large for any one person to master it all. Not even Sid knows everything about Civ any more!" And it was true! My fellow group leaders were very impressive. We soon agreed on the things we had in common and formed a team. Early in the session, when the first batch of testers were being brought in, the three group leads got to play some multiplayer against one another and against the Firaxians. There were two tests that really stick out in my mind. In both cases, they gave us the short straw. One was a [three-on-two] game. Jesse (Friedrich Psitalon) started closer to the three Firaxians. They ganged up on him and eliminated him early on, and I was too far away to get any help to him in time. Who was in that game? I think it was Soren, Ryan Meier (Sid's son), and Eric MacDonald? So then it was the three of them left against me, all alone. I held them off for the rest of the evening! Soren was the most aggressive. He tried over and over to attack me. They had a massive tech lead, a lead in material, more cities, more everything, but I managed to build The Oracle Great Wonder in a back line city and used it to nab critical military technology that kept me in the game. The next test was almost the same deal. That time it was [four-on-three]. On our side we had [Lead P]roducer Jesse Smith. Soren's side had four players. Again, Friedrich got put in an awful start location, surrounded by all four of the other team, with Jesse Smith and I tucked off in the west. It took all four of them combined to beat down Friedrich! And he still rebuffed their first two attack waves! I then predicted to Jesse (my only remaining teammate) what would happen next. I knew Soren would try to press the advantage and eliminate one of us, try to end the game quickly. We tried to prepare, but it wasn't enough. Jesse was eliminated, too, and then it was 4 on 1. They tried to eliminate me, too, but I stacked myself to the gills with military units and they could not do it. So they settled back and teched their way to victory. We learned all sorts of lessons from these two tests, including the need for improving the balance of start locations for team games. Every test would bring something new to the surface. Soren would always listen to the feedback, then make his calls on what to change, or what to try. We tried a lot of "experimental" things along the way. The ones that didn't pass muster got changed or removed.
Solver:	What was the CivIV play session like later? More testers were brought into it – what else had changed and what did the testers accomplish during the final few months of CivIV development?
Sirian:	Testers came and went, for the most part. I know a lot of fans have said they would have given anything to participate in something like this, but in part that is a case of not knowing what you are really in for. Try to imagine. When a game is in development, the rules will change from one build to the next. Parts that were missing get worked on and added. Concepts that failed get cut. Imbalances are identified and changed. You are not playing the same game from week to week. Sometimes, the balance would shift radically. And... there would be bugs. Bugs could cause all manner of chaos. Occasionally, a whole build would just be... broken. No productive testing that week! Usually things worked properly, but you just never know. Is something a bug or a feature? It could be hard to tell. We maintained separation between the internal development and the play session. Sometimes the play session members would just be flying in the dark! So if you came in to test, you volunteered for a wild ride! Only a handful of folks proved to have the stomach to ride that ride over and over for months on end. A lot of our testers would exhaust themselves and need a break, or would be called back to real life responsibilities and have to stop. As many as half our testers simply washed out completely. Things moved too fast, or the ratio of fun to work was not what they expected. A few did not like some of the changes. We also had a need for fresh eyes. After you've been exposed to so many different builds, some of the details start to blur. Bringing in new people would give us a chance to get first impressions and feedback untainted by earlier builds, by things that got changed, added or removed. A healthy mix of veteran testers who do remember all the experiments and changes and what was already tried before and did not work out well, along with fresh eyes, gave us an added dimension of feedback. Checks and balances! When the bulk of the play session agreed on a particular point, Soren almost always went with it. In particular, for game balance issues, if both Friedrich and I signed off on something, Soren would trust it. If it was OK both for single player and multiplayer, and it survived testing, it was good. When the play session disagreed... well, then it got more challenging. Often what would happen is that Soren would consider all points of view... and not go with any of them! He would “send it back” and make us look again, work harder, try to see if we missed some possibilities. That was frustrating, sometimes. I really came to appreciate it by the end, though. Patience! Soren had immense patience. He was willing to let things cook a while longer if they didn't taste just right, as yet. From the tester's point of view, though, this could all be a trying process! Different folks contributed at different times. A few stalwarts were MVPs all the way through. A lot of fan voices from many different interests and communities got the chance to voice their opinions, and they were heard! The strategy behind the session was to limit the number of voices, to hand pick a representative sample from every corner of Civ fandom and triangulate the feedback. I believe this approach worked as intended. I have seen other games try to involve fans by dealing with them out in the open. Too often, that devolves into the loudest crusaders getting what they want, at the expense of quiet but rational voices who go unheard. Although I was a paid worker, I have been a fan volunteer in the past. Nearly all of us had done this before, at least once. I made every effort to improve things for our volunteer testers, to ensure they had a good time. Sometimes the internal folks would more or less forget that the testing crew were volunteers. I would not let them forget! I would step in and urge that the reins be loosened, that the fans be left to guide their own ships, and that the paid workers step in to the areas the fans weren't covering. The idea was to let the fans do the fun stuff, basically. After all, that -was- their payment! That and getting a better game to play in years to come.

PAGE 1 | PAGE 2 | PAGE 3