Bertie
search
auto_awesomeActioninventory_2Consultant review
Action 4 · Task 373 · Group 20

Execute Calibration Session and Produce Master Framework

Facilitate a structured calibration session with key evaluators using test venture cases to align scoring outcomes. Finalise the definitive Evaluator Calibration Guide and baseline score normalisation rules.

Objective

Completing this action delivers the operational artefact that enforces consistent scoring across all active evaluators. It converts individual subjective judgment into a repeatable, institution-grade evaluation system.

What's expected from the founder

Publish a completed Evaluator Calibration Guide alongside a signed-off score normalisation matrix tested against sample ventures. Provide documented evidence of evaluator score convergence within an acceptable statistical tolerance.

psychologyBertie consultant stress-test

Five questions an expert would ask when reviewing your output

Use these to challenge assumptions, pressure-test your logic, and check the quality of this action's output in the context of the parent task and wider venture development.

  1. 1

    What degree of score variance remained across evaluators after the calibration exercise, and why?

  2. 2

    How did the calibration session resolve fundamental disagreements on high-risk, high-reward ventures?

  3. 3

    Why are you confident that this normalisation matrix will hold when applied to live deal flow?

  4. 4

    How does the final artefact handle situations where an evaluator lacks domain expertise in a niche technology?

  5. 5

    What explicit decision rules were established to override outlier scores during live panel sessions?