client questioning your test results?

[link]


my test documentation should always include: 


a. test map: trace-referenced, logged, and updated to include all the scenario branches that were actually tested at the time of testing;

b. test evidences: vid captures, screen captures, actual app outputs (images, jsons, txt files, etc), app exe file, that prove how the app behaved at the time of testing -- any and all relevant artefacts that captured the app's behaviour;

c. bug/issue reports: replete with STRs (steps to replicate), evidences, paired against expected results;

d. the app build or version number tested; 

e. the platform tested;

d. Pass/fix statement on the test report: indicating (a), (b), (c), (d), (e);


this is why, broadly speaking, all testing should have these elements to provide a thorough proof that testing was executed to properly evaluate the software under test. Any misses will be self evident with this method of documentation. 

Comments

Post a Comment

Popular posts from this blog

updated: Regression Tests as a normal part of day to day exploration / Regression Testing

Image
  Even before i heard of CICD, i had already coined the term 'continuous regression' in our office. For me it means 'testing for regressions' are not a specific activity, but incorporated into the testing/exploring process itself.  In the Hypothesis of Test Phase Space, every dot in that space represents a state of the Test (one dot an input, the other dot an output). And every line connecting those two dots is a Test Path. Each line comprises of multiple other dots between the two end-dots: these are potential branch points to other dots (linking to misbehaviour-outputs, or linking to other test paths) such that while exploring current code changes/fixes it is inevitable that some of these code changes may/might be linked to other parts of the product. It is in such context that i couch continuous regression testing.  Continuous modular regression testing should be part of normal exploratory testing. How fast and how thorough should these tests be? As speedy as the deg...
Read more

The Quality Advocate (Test supervisor / manager / lead)

Image
A Quality Advocate isn't / shouldn't be about auditing the work of your fellow testers.  The work of a Quality Advocate should be to check the current testing procedures vis-a-vis the product/prototype under test.  When you have testers testing under you / for you, the supervisor's role is not to check if the tester tested the product properly. Because the tester has already been coached to be critical-thinking, and exploratory -- so assuming his/her evaluative report shows exactly that -- then rather the sup's job is to see if there's an expanded way to model the product under test, to give a more justified evaluation.  What if we find there is nothing more to expand? Then either (a) you, as a supervisor, have failed in your task; or (b) your test team is above par excellent.   note: 'justified' :   General Context: In a general sense, "justified" can mean that something is reasonable, logical, or well-founded. For example, "Her concerns were...
Read more

Test Report1

Image
================ Test Report --------------------------- Product info: (what is it we are testing? ; complete info) Product Eval: (what can it do ; can't do ; suitable for what ; product claims vs product capabilities) Test_Map, Results: + behaviours, - behaviours, AssocRisks -- in table form Supporting_Artefacts(evidences[vids,pics,files,urls,etc])
Read more