client questioning your test results?

[link]


my test documentation should always include: 


a. test map: trace-referenced, logged, and updated to include all the scenario branches that were actually tested at the time of testing;

b. test evidences: vid captures, screen captures, actual app outputs (images, jsons, txt files, etc), app exe file, that prove how the app behaved at the time of testing -- any and all relevant artefacts that captured the app's behaviour;

c. bug/issue reports: replete with STRs (steps to replicate), evidences, paired against expected results;

d. the app build or version number tested; 

e. the platform tested;

d. Pass/fix statement on the test report: indicating (a), (b), (c), (d), (e);


this is why, broadly speaking, all testing should have these elements to provide a thorough proof that testing was executed to properly evaluate the software under test. Any misses will be self evident with this method of documentation. 

Comments

Post a Comment

Popular posts from this blog

Pyramid, Honeycomb, Feather Testing Models

Image
How much time should be allotted to each level of test activity (unit test, integration test, system test, end-to-end test)? it varies; but definitely just enough to get the job done to an agreed level of Acceptance Credence.  A model that is not adaptable to the task at hand constricts development, and needlessly constricts the actors themselves(coder, tester) -- such that, either  (a) the team starts to get coerced to follow the model, sacrificing any innate efficiency; or  (b) the team ignores the model and  lets the nature of the task dictate the most efficient work flow.  The nature of the test_paths themselves would provide the impetus  to place  emphasis on either  (c)   integration tests (e.g. majority of the test_paths are integration subsystems); (d) a balance of unit tests and end-to-end tests (e.g. the product is simple, or a self-contained supersystem); or (e) a balance of unit, integration, and comprehensive systemic tests ...
Read more

The phenomenon of QA(quality advocacy) in Test-PhaseSpace

Image
  Quality Advocacy                      \___ 3rd tier/level --- designers,  coders                      \___ 2nd tier/level --- quality advocates                      \___ 1st   tier/level  --- grassroots testers  ================= definitions:  ================= ..Quality [ Q ] = the state of the product (high or low) at any given time (m).  ..QA [ Quality Advocacy ] = the monitoring/measuring of quality to know quality ; unless it is monitored/measured, quality [ Q ] cannot be known. Quality Advocacy extends to all 3 tiers/levels.  ..QA [ Quality Assurance ] =  working together  as a  Development Team (coders + testers + proj.mgr + designers + etc.) which will always deliver a specific level of quality. ..software tester = 1st level, grassroots interactor [someone...
Read more

The Quality Advocate (Test supervisor / manager / lead)

Image
A Quality Advocate isn't / shouldn't be about auditing the work of your fellow testers.  The work of a Quality Advocate should be to check the current testing procedures vis-a-vis the product/prototype under test.  When you have testers testing under you / for you, the supervisor's role is not to check if the tester tested the product properly. Because the tester has already been coached to be critical-thinking, and exploratory -- so assuming his/her evaluative report shows exactly that -- then rather the sup's job is to see if there's an expanded way to model the product under test, to give a more justified evaluation.  What if we find there is nothing more to expand? Then either (a) you, as a supervisor, have failed in your task; or (b) your test team is above par excellent.   note: 'justified' :   General Context: In a general sense, "justified" can mean that something is reasonable, logical, or well-founded. For example, "Her concerns were...
Read more