iSQI CTFL-PT Exam - Topic 6 Question 31 Discussion

Question

You have been asked to test the performance of a software and system update to a North American mobile trivia quiz game application. The quiz app is available on major app stores serving English-speaking users. The number of concurrent users planned for is 5 million Winning quiz participants are promised a share of an allocated cash prize. Once participants start playing, their quiz sessions must not be lost especially if all previous questions were answered correctly. The application owner wants to ensure that 99% of quiz sessions are maintained (up to 4 95 million sessions) until the last answer has been successfully recorded.

Before your team begins testing, the team needs to present the performance test plan to business stakeholders

Which of the following sets of plan attributes would you recommend be part of this presentation?

Roslyn · Accepted Answer

C) Since the client uses an asynchronous REST web services based sessions back to the QUZRAC services cluster, the load profile will consist of virtual users running programmed load scripts based in C# that will randomly select the correct answer following a normal distribution   thus modeling the typical success distribution among users  The strict 4 second cap on user responses will translate into think times that are randomly selected between 1/2 second and just under 4 seconds using a uniform distribution

Roslyn · Answer

A) During the actual quiz, users logon gradually but play simultaneously. The load injection of quiz responses will not ramp up. therefore nearly 5 million quiz responses will need to be generated in a 4 second time frame with no opportunity to reach steady state.  Test plans have been discussed with Operations Development and Data Center representatives and we believe the tests will tax the quiz application consistent with actual use. The JMeter monitoring system has been tuned however, with 5 million service requests over 4 seconds there is concern that the services queue measurements may be off by 25%

Roslyn · Answer

B) There is a less then10% chance of missing a scheduled quiz session due to performance test activities, which will likely cost the company $50,000 in lost sponsor revenue and brand damage  To avoid this risk, we are scheduling Operations and Development representatives to be available during test execution  We will require that the RAID 5+1 LUNs associated with the database server cluster be broken at 22 30 and the RAID 5+1 LUNs restored, ensuring the offline set mirrors back to the RAID 5 set that was under test, at 03:00

Roslyn · Answer

D) If the quiz application drops more than 1% of quiz sessions, users will likely write bad reviews on app stores and other gaming sites
Performance tests will verify that system can support at least 4 95 million users   Planned tests, which include the simultaneous sessions test, will cost $350,000   Since only two quizzes are scheduled a day, performance tests on the production system during the 'down' times should be representative of what users experience. There is a 10% chance that the production system will not restored in time for a quiz.