Don’t they test software anymore?

This is not a rant about the constant patching of my PS3 everytime I turn the thing on, nor about the minor updates hear and there in various operating systems and platforms. Well actually it might be, but its not intended to be rant at all.
Whilst playing the wonderful Just Cause 2 on the Xbox I bumped into a few bugs and glitches. They all happened around the large radio antenna dish up in the mountains.
The first saw me accidentally get inside of the buildings. The buildings tend to be texture walls to climb so once you are inside they seem to be invisible as there is not texture to render. This was amusing until I realized I could not get out at all. I was not able to spoof an evac and had no ammo to self harm so I had to quit 🙁 Still these things happen.
I then popped back to the same place and found a little glitch with the top edge of a building.
Just Cause 2 Glitch
i.e. I am standing on nothing due to some odd collision detection.
For me I do not regard these are terrible flaws but it does raise the question of how, with such large expansive experiences it is ever possible to get enough test coverage and QA in place. Just Cause 2 has 400 sq miles of terrain, trees, building water. Vehicles moving around. Clearly the physics engine can be tested with a relative few vectors but when the place is built by hand, building placed its almost impossible to not have a few errors in place. Equally with a free roaming game its hard to tell when something is a bug.
I had driven a speed boat straight at a yacht. The speed boat was forced underwater with me still driving it, I had a submarine in effect. That is less of a bug a more an unintended consequence, the thing was it then became my mission to try and recreate it, which I eventually did.
Testing and coverage is actually not an instinctive thing to do as I have found with some clients. Where an atomic action happens with the same start and end conditions, but where it is held in a different part of a flow or sequence each time it is not always so necessary to run a test for every permutation and combination. Test the changes, test the extremes and test the critical path.
The software I used to build way back for internal corporate systems testing was a straightforward job in many respects, as we started to get more and more service, more and more permutations and many more functions it clearly has got a lot harder.
It is why I am not surprised that some tests really don’t get done in favour of patch later over the web, though at the same time I suspect some don’t get done because “hey we can patch it later”.
Anyway I am not moaning about Just Cause 2 as I am still in awe of its size and scale, just as I am with GTA IV and will no doubt be with Red Dead Redemption. Free roaming FTW!

3 thoughts on “Don’t they test software anymore?

  1. Maybe just focus on testing the save system first so at least you can always revert out of a stuck situation 🙂

  2. 🙂 Luckily in this situation the save system was pretty good. Again though with an open ended free roaming environment which has potential chaotic overtones of the smallest change having a huge knock on effect what gets saved and when can become incredibly complex.

  3. Pingback: Read Dead Redemption – it is what we miss that makes it so good « Life at the Feeding Edge

Leave a Reply

Your email address will not be published. Required fields are marked *

tIXR f

Please type the text above:

This site uses Akismet to reduce spam. Learn how your comment data is processed.