Game Testing - Methodology

Methodology

There is no standard method for game testing, and most methodologies are developed by individual video game developers and publishers. Methodologies are continuously refined and may differ for different types of games (for example, the methodology for testing an MMORPG will be different from testing a casual game). Many methods, such as unit testing, are borrowed directly from general software testing techniques. Outlined below are the most important methodologies, specific to video games.

• Functionality testing is most commonly associated with the phrase "game testing", as it entails playing the game in some form. Functionality testing does not require extensive technical knowledge. Functionality testers look for general problems within the game itself or its user interface, such as stability issues, game mechanic issues, and game asset integrity.

• Compliance testing is the reason for the existence of game testing labs. First-party licensors for console platforms have strict technical requirements titles licensed for their platforms. For example, Sony publishes a Technical Requirements Checklist (TRC), Microsoft publishes Technical Certification Requirements (TCR), and Nintendo publishes a set of "guidelines" (Lotcheck). Some of these requirements are highly technical and fall outside the scope of game testing. Other parts, most notably the formatting of standard error messages, handling of memory card data, and handling of legally trademarked and copyrighted material, are the responsibility of the game testers. Even a single violation in submission for license approval may have the game rejected, possibly incurring additional costs in further testing and resubmission. In addition, the delay may cause the title to miss an important launch window, potentially costing the publisher even larger sums of money.

The requirements are proprietary documents released to developers and publishers under confidentiality agreements. They are not available for the general public to review, although familiarity with these standards is considered a valuable skill to have as a tester.

Compliance may also refer to regulatory bodies such as the ESRB and PEGI, if the game targets a particular content rating. Testers must report objectionable content that may be inappropriate for the desired rating. Similar to licensing, games that do not receive the desired rating must be re-edited, retested, and resubmitted at additional cost.

• Compatibility testing is normally required for PC titles, nearing the end of development as much of the compatibility depends on the final build of the game. Often two rounds of compatibility tests are done - early in beta to allow time for issue resolution, and late in beta or during release candidate. Compatibility testing team test major functionality of the game on various configurations of hardware. Usually a list of commercially important hardware is supplied by the publisher.

Compatibility testing ensures that the game runs on different configurations of hardware and software. The hardware encompasses brands of different manufacturers and assorted input peripherals such as gamepads and joysticks.

The testers also evaluate performance and results are used for game's advertised minimum system requirements. Compatibility or performance issues may be either fixed by the developer or, in case of legacy hardware and software, support may be dropped.

• Localization testing act as in-game text editors. Although general text issues are a part of functionality testing, QA departments may employ dedicated localization testers. In particular, early Japanese game translations were rife with Engrish, and in recent years localization testers are employed to make technical corrections and review translation work of game scripts - catalogued collections of all the in-game text. Testers native to the region where a game is marketed may be employed to ensure the accuracy and quality of a game's localization.

• Soak testing, in the context of video games, involves leaving the game running for prolonged periods time in various modes of operation, such as idling, paused, or at the title screen. This testing requires no user interaction beyond initial setup, and is usually managed by lead testers. Automated tools may be used for simulating repetitive actions, such mouse clicks. Soaking can detect memory leaks or rounding errors that manifest only over time. Soak tests are one of the compliance requirements.

• Beta testing is done during beta stage of development. Often this refers to the first publicly available version of a game. Public betas are effective because thousands of fans may find bugs that the developer's testers did not.

• Regression testing is performed once a bug has been fixed by the programmers. QA checks to see whether the bug is still there (regression) and then runs similar tests to see whether the fix broke something else. That second stage is often called "halo testing"; it involves testing all around a bug, looking for other bugs.

• Load testing tests the limits of a system, such as the number of players on an MMO server, the number of sprites active on the screen, or the number of threads running in a particular program. Load testing requires either a large group of testers or software that emulates heavy activity. Load testing also measures the capability of an application to function correctly under load.

• Multiplayer testing may involve separate multiplayer QA team if the game has significant multiplayer portions. This testing is more common with PC games. The testers ensure that all connectivity methods (modem, LAN, Internet) are working. This allows single player and multiplayer testing to occur in parallel.