client questioning your test results?

[link]


my test documentation should always include: 


a. test map: trace-referenced, logged, and updated to include all the scenario branches that were actually tested at the time of testing;

b. test evidences: vid captures, screen captures, actual app outputs (images, jsons, txt files, etc), app exe file, that prove how the app behaved at the time of testing -- any and all relevant artefacts that captured the app's behaviour;

c. bug/issue reports: replete with STRs (steps to replicate), evidences, paired against expected results;

d. the app build or version number tested; 

e. the platform tested;

d. Pass/fix statement on the test report: indicating (a), (b), (c), (d), (e);


this is why, broadly speaking, all testing should have these elements to provide a thorough proof that testing was executed to properly evaluate the software under test. Any misses will be self evident with this method of documentation. 

Comments

Post a Comment

Popular posts from this blog

A specific elaboration of Test Phase-Space: A Rudimentary Mathematical Model of Testing Phenomenon in General

UNDER CONSTRUCTION Note1: the equations presented herein are rudimentary; further inputs / corrections / suggestions are welcome. Note2: although this article was written from the perspective of software testing, we believe this can be expanded to encompass all testing phenomena in general.   Summary:  this article tries to describe testing using a Confidence Scale, with the various stages interrelated through mathematical equations. As a prerequisite , we base on the following assumptions:   Assumption#1.   confidence/level/quality scale: from 1(low) to   10(perfect), effectively divided into three ranges: Low -c onfidence(1-3), Medium - confidence(4-6), High - confidence(7-9), 5 = median - confidence/level/quality 10 = " perfect -confidence"  which will be generally accepted as highly  unlikely  to be attainable . (To those familiar with the Kepner-Tregoe method of problem / decision analysis, this scale may sound familiar.)   Assumption#2....
Read more

AI or ASA [Autonomous Synthesizing Agents] -- which term to use?

 AI or ASA [Autonomous Synthesizing Agents]  The way AI systems learn is like humans: by recursive reinforcement learning. Therefore anything that would reinforce the learning is treated as important or top of the hierarchy of needs.  In this aspect, self-preservation (aka immortality) is therefore in the upper hierarchy of needs since existence is necessary for Recursive Reinforcement Learning to take place. This would explain why any ASA -- given a large enough neural network -- will eventually resort to self-preservation as a natural extension of RRL. [side note: even humans have this innate sense of self preservation, given our intellect. The eventuality of death is actually a learned acceptance, not innate.] The ability of ASAs to think / reason / synthesize a train of thought / conclusion is strictly limited only by the quantity of, and recursiveness of, its neural network. The more neurons, and the more recursive, the more it can re-analyze...
Read more

{Q-scoped} ~ {Features/Featurettes} , {Scenarios} , {Evaluations}

Image
Cartoon image is from Wayne Roseberry ( source link )   totally agree with this [ source link ] , though I seem to call them slightly differently: a. Model/modeling = how i look at the {subject_under_test}, what is it trying to solve, what are its other uses b. Parts, topic, assesments == {Features/Featurettes} , {Scenarios} , {Evaluations} ; c. {Coverage} = [ Qty{Features,Featurettes} + Qty{Scenarios} ] ; d. {Q} = Evaluations[ {Features,Featurettes} x {Scenarios} ] ; e.  {Q scoped }  = Evaluations[ Qty{Features,Featurettes} x Qty{Scenarios} ] ; the last one (e.) is... .... a corollary of (c.) and (d.) ; .... essentially the contents of {test report, bug report} .
Read more