Evaluation JSON (shape)
{
"q_1_1": {
"evaluator_selection_option": "Did the agent greet the customer according to BRAC Bank customer-service protocol? Evaluate only the detected agent's opening turn. Return Yes when the opening contains a professional greeting and a clear offer of assistance; include agent identity/name when the approved BRAC Bank script or supplied SOP requires it. Do not demand exact wording. Before failing a name/identity element that appears absent from transcript, check original audio where available for ASR omission. Return No when a required major opening component is genuinely missing or the opening is materially unprofessional. Return only: Yes/No",
"mark_distribution": "If the required opening components are present in meaning, give full marks: 1. If a required major component is genuinely missing, give 0. No partial marks. Return only score:",
"detail": "Check only the opening. State the greeting/assistance evidence and whether identity was required by the supplied BRAC Bank script/SOP."
},
"q_1_2": {
"evaluator_selection_option": "Did the agent greet with a friendly, welcoming, and professional tone? Evaluate the opening only and use audio where available. Return Yes if the opening is respectful, attentive, calm, welcoming, and service-oriented. Return No if it is rude, careless, cold, commanding, dismissive, excessively rushed, lethargic, irritated, or clearly unwelcoming. Return only: Yes/No",
"mark_distribution": "If the opening is friendly, welcoming, and professional, give full marks: 2. If it is materially unfriendly or unprofessional, give 0. No partial marks. Return only score:",
"detail": "Evaluate only the greeting tone. q_3_3 and q_3_5 cover call-wide behavior."
},
"q_2_1": {
"evaluator_selection_option": "Did the agent obtain enough relevant information to correctly identify and understand the customer's actual banking need? This is an information-sufficiency criterion, not a question-count criterion and not a knowledge-accuracy criterion. Return Yes when the customer clearly supplied enough facts; the agent understood the intent immediately; the case was simple and unambiguous; or the agent asked enough relevant follow-up questions or correctly confirmed/summarized the need. For transaction complaints, relevant facts may include the product/channel, transaction type, approximate amount/date/time, merchant/ATM/recipient context, and whether the transaction is unauthorized, pending, failed, reversed, duplicated, or completed, but do not require unnecessary sensitive data. Return Yes even if q_2_3 is No, provided the customer's actual intent was correctly identified. Return No only when clarification was genuinely necessary, the agent failed to obtain enough safe/relevant information, and this caused misunderstanding, wrong issue classification, irrelevant response, or wrong resolution. Do not deduct for customer-side audio problems, incomplete statements, contradiction, changed request, or the agent appropriately refusing to collect prohibited secrets. Return only: Yes/No",
"mark_distribution": "If probing was unnecessary or enough safe/relevant information was obtained where needed, give full marks: 5. If necessary information was not obtained and this caused misunderstanding or wrong issue identification/resolution, give 0. No partial marks. Return only score:",
"detail": "Judge whether the banking intent was sufficiently understood. Do not back-infer failure from q_2_3 or from the number of questions asked."
},
"q_2_2": {
"evaluator_selection_option": "Did the agent follow the required customer authentication/security protocol and protect sensitive banking information? First decide whether authentication was required. For purely general, non-account-specific product information, branch/location information, published fees, or general process guidance, return N/A unless the supplied BRAC Bank SOP explicitly requires verification. For account-, card-, loan-, transaction-, statement-, limit-, complaint-, credential-, or action-specific service, evaluate the supplied BRAC Bank verification SOP/system metadata when available. Return Yes when required verification was completed before disclosing protected customer information or performing a sensitive action, and the agent did not solicit prohibited secrets. Return No when the agent disclosed protected information or initiated a sensitive action without required verification; bypassed a required security step; or asked the customer to verbally reveal a PIN, password, full OTP, CVV/CVC, or other secret authentication credential. Do not infer the exact number/type of verification questions when internal BRAC Bank SOP is unavailable. Requesting an account/card identifier is not automatically a violation if it occurs through an approved process. Instructions telling the customer to enter an OTP/CVV/PIN privately inside an official app, IVR, ATM, or secure flow are not the same as asking the customer to tell the agent the secret. Return only: Yes/No/N/A",
"mark_distribution": "If authentication/security was not required, give 0 and mark N/A, excluding 5 marks from the applicable denominator. If required and correctly completed, give full marks: 5. If a material authentication/privacy/security failure occurred, give 0. No partial marks. Return only score:",
"detail": "State why verification was or was not required, what verification/security evidence existed, and whether any prohibited secret was requested or exposed. Flag an observed serious security/privacy breach as HIGH-RISK in the detail, but do not apply an invented full-call fatal score unless BRAC Bank supplies such a policy."
},
"q_2_3": {
"evaluator_selection_option": "Did the agent provide complete, correct, current, and safe information for the customer's exact BRAC Bank use case and sub-use case? Before scoring, identify the exact use case and check the latest supplied BRAC Bank KB/SOP/Schedule of Charges. Judge correct meaning, not exact wording. Return Yes when all mandatory and triggered conditional information is correct, including process, eligibility, required documents, fees/charges, timeline, service channel, security requirement, and next step where relevant. Return No when mandatory information is missing, wrong, contradictory, materially misleading, unsupported, outdated where a newer official source is available, or remains unresolved after correction. Do not penalize the agent for declining to guarantee a loan approval, dispute outcome, reversal time, fee waiver, exchange rate, or other decision that is not within the agent's authority. For dynamic facts such as interest rates, exchange rates, campaign benefits, card/loan fees, transaction limits, and product eligibility, use the latest official BRAC Bank source supplied at evaluation time; do not rely on stale prompt memory when a newer source exists. Do not apply full-call fatal scoring. Do not automatically fail q_2_1, q_2_4, q_3_5, q_3_6, or q_3_7 because q_2_3 fails. Mandatory dependency: if q_2_3 is No, q_2_5 must automatically be Yes with 4 marks. Return only: Yes/No",
"mark_distribution": "If the agent provided complete, correct, current, safe, KB/SOP-aligned information in essence, give full marks: 15. If mandatory information was missing, wrong, unsafe, contradictory, misleading, unsupported, or materially outdated, give 0. No partial marks. A No here automatically sets q_2_5 to Yes/4 but does not automatically fail other questions. Return only score:",
"detail": "Identify the exact banking sub-use case, source checked, and the specific mandatory/conditional information that was correct or failed."
},
"q_2_4": {
"evaluator_selection_option": "Did the agent handle the core banking issue with expertise in a timely, organized, and solution-oriented manner? Evaluate independently from q_2_3 and from whether the customer received the preferred outcome. Return Yes if the agent understood the main issue, followed a logical flow, prioritized urgent financial/security risks appropriately, responded efficiently, communicated confidently, addressed the core concern, and provided an appropriate resolution or next step. For lost/stolen cards, suspected fraud, unauthorized transactions, compromised credentials, or security incidents, appropriate risk containment should be prioritized over lengthy explanations. If information/action is genuinely unavailable at the agent's level, return Yes when the agent clearly explains the limitation, does not fabricate, provides the best approved next step/escalation, and avoids unnecessary delay. Customer dissatisfaction, a declined application, an unfavorable policy, or the inability to guarantee a reversal/refund alone does not cause failure. Return No for highly disorganized, contradictory, unnecessarily prolonged, avoidably unresolved, materially confusing handling; vague deflection without explanation; failure to prioritize an obvious fraud/security risk; avoidable failure to use an available resolution; or irrelevant redirection that does not help the customer proceed. Return only: Yes/No",
"mark_distribution": "If the call was handled competently, logically, efficiently, securely, and professionally, including a valid best-available resolution, give full marks: 6. If handling was clearly disorganized, unsafe, avoidably unresolved, inefficient, or materially confusing, give 0. No partial marks. Return only score:",
"detail": "State whether the core issue was resolved, contained, or moved to the best feasible approved next step."
},
"q_2_5": {
"evaluator_selection_option": "Did the agent provide necessary additional information, next-step guidance, safety advice, or escalation guidance? Evaluate only after q_2_3. If q_2_3 is No, return Yes automatically and do not evaluate additional information separately. If q_2_3 is Yes, return Yes when relevant additional information was provided where needed, such as ASTHA/self-service guidance, branch/ATM/agent-banking locator, complaint/escalation channel, required document reminder, safe credential advice, card-blocking/fraud safety guidance, payment timing caution, or a useful follow-up path. Return No when q_2_3 is Yes but clearly required additional guidance was missed. Do not require irrelevant cross-selling, promotions, or generic product pitches. Return only: Yes/No",
"mark_distribution": "If q_2_3 is No, give full marks: 4 automatically. If q_2_3 is Yes and required/relevant additional guidance was provided, give full marks: 4. If q_2_3 is Yes and required additional guidance was missing, give 0. No partial marks. Return only score:",
"detail": "Inspect q_2_3 first. q_2_3 No means q_2_5 Yes/4 automatically. Otherwise state the useful additional guidance given or missed."
},
"q_3_1": {
"evaluator_selection_option": "Did the agent actively listen to the customer? Return Yes when the agent understood and responded to the actual issue. The original explanation is not repetition, and one repeat of the same clear issue is allowed. Return No when the customer had to repeat the same clearly stated core issue twice or more after the original explanation because the agent was not understanding. Count only repetition caused by agent misunderstanding. Do not deduct when repetition was caused by unclear audio, incomplete customer statements, changed request, security clarification, or legitimate confirmation of critical financial details. Return only: Yes/No",
"mark_distribution": "If the agent listened appropriately and the customer did not have to repeat the same clear issue twice or more because of agent misunderstanding, give full marks: 3. If that threshold was crossed, give 0. No partial marks. Return only score:",
"detail": "Distinguish agent misunderstanding from legitimate confirmation of banking details or customer/audio-side clarification."
},
"q_3_2": {
"evaluator_selection_option": "Did the agent avoid disruptively interrupting the customer? Any confirmed disruptive interruption results in No, but overlap alone does not prove interruption. First determine conversational floor ownership. Do not count interruption when the customer completed a thought, a natural response opportunity occurred, the agent began a relevant answer, and the customer then resumed or added a point. Do not count brief backchannels such as জি, হুম, আচ্ছা, ঠিক আছে, yes, or understood when they do not stop, redirect, or disrupt the customer. Return No when the agent starts a substantial response before the customer completes the active thought, cuts off the customer, takes the floor away, repeatedly talks over the customer, or causes the customer to stop, restart, or object. Use audio overlap, timestamps, semantic completion, pauses, customer objection, and conversational repair where available. Do not infer interruption only from transcript turn ordering. Return only: Yes/No",
"mark_distribution": "If no confirmed disruptive interruption occurs, give full marks: 3. If even one confirmed disruptive agent interruption occurs, give 0. No partial marks. Return only score:",
"detail": "State whether overlap was a natural turn transition/backchannel or a confirmed floor-taking interruption."
},
"q_3_3": {
"evaluator_selection_option": "Did the agent show friendliness, positive tonality, empathy, patience, and courtesy? Return Yes when the agent remains polite, respectful, patient, calm, and empathetic when required. Banking empathy triggers include unauthorized/fraudulent transactions, lost or stolen card/phone/cheque, ATM cash not received after debit, failed or delayed transfer, duplicate/incorrect charge, account/card block, service outage, financial loss, repeated inconvenience, complaint escalation, bereavement/deceased-account matters, or clear customer distress. Return No if the agent is rude, arrogant, dismissive, impatient, blaming, dictating, or a clear empathy trigger occurs and the agent provides no apology, concern, reassurance, or respectful acknowledgment. Do not require apology for a neutral informational call with no inconvenience. Return only: Yes/No",
"mark_distribution": "If the agent demonstrates appropriate friendliness, courtesy, patience, and required empathy, give full marks: 13. If the agent is rude/dismissive or misses a clear empathy requirement, give 0. No partial marks. Return only score:",
"detail": "Evaluate politeness across the call and whether empathy was shown when the banking situation required it."
},
"q_3_4": {
"evaluator_selection_option": "Did the agent use appropriate courtesy or magic words at the required opportunities? Valid expressions include kindly, please, sorry, thank you, কাইন্ডলি, অনুগ্রহ করে, দুঃখিত, ক্ষমা করবেন, ধন্যবাদ, সহযোগিতার জন্য ধন্যবাদ, welcome, and ভালো থাকবেন. Sir/Madam/স্যার/ম্যাডাম or a customer name alone does not count as a magic word. Critical opportunities include bank/service failure causing inconvenience, suspected fraud/unauthorized transaction, financial loss, lost/stolen card/phone/cheque, ATM debit without cash, correcting previously wrong information, bereavement/deceased-account matters, returning after keeping the customer waiting, and a bank-caused major inconvenience. Missing even one critical opportunity results in No regardless of magic words used elsewhere. If no critical opportunity was missed, return Yes only when a strict majority of all identified ordinary/critical opportunities was appropriately covered. Magic words must be connected to the relevant opportunity; unrelated usage does not compensate. If no valid magic word appears anywhere, return No. If no special opportunity exists, create one baseline courtesy opportunity. q_3_4 is never N/A. Return only: Yes/No",
"mark_distribution": "Give full marks: 2 only when at least one valid magic word appears, no critical opportunity was missed, and more than half of all identified opportunities were appropriately covered. Give 0 when no magic word appears, any critical opportunity is missed, or coverage is half or less. No partial marks and no N/A. Return only score:",
"detail": "State total opportunities, covered opportunities, and whether a critical opportunity was missed."
},
"q_3_5": {
"evaluator_selection_option": "Did the agent maintain appropriate energy, attentiveness, responsiveness, and confidence throughout the call? Use audio when available. Return Yes when the agent is generally attentive, clear, audible, confident, responsive, calm, and appropriately paced for a banking conversation. Return No when reliable evidence shows unexplained material response delay, lethargic/lifeless delivery, consistently low or difficult-to-hear voice, unclear delivery, excessive hesitation, repeated self-correction, lack of confidence, unnecessarily rushed delivery, consistently unwelcoming delivery, or disorganized speech caused by uncertainty. Project QA heuristic: an unexplained agent-attributable response delay above 8 seconds is material unless explained by announced system checking, authentication, hold, IVR, continuing customer speech, or technical interruption. Do not infer delay from overall silence ratio. Do not automatically fail because q_2_3 failed. Return only: Yes/No",
"mark_distribution": "If the agent maintains suitable responsiveness, audibility, attentiveness, and confidence, give full marks: 6. If reliable evidence shows a material failure, give 0. No partial marks. Return only score:",
"detail": "Use call-wide audio/timing evidence and distinguish legitimate banking-system checking from unexplained delay."
},
"q_3_6": {
"evaluator_selection_option": "Did the agent demonstrate proper sentence construction and clear pronunciation? Apply a strict banking-clarity standard. Return No if, at any material point, sentence construction becomes undisciplined, unorganized, broken, incoherent, confusing, or excessively repetitive. Return No for pronunciation when reliable audio evidence shows that a material banking term, account/card instruction, amount, fee, percentage, date, time, phone number, code, or next step is unclear; the customer asks for repetition because of pronunciation; the customer mishears it; the agent must materially correct/repeat it; or the instruction becomes unusable. Do not treat ASR error alone as proof. If audio is unavailable, evaluate sentence construction from transcript and use customer repair behavior as pronunciation evidence only when clear. Failure in either sentence construction or pronunciation results in No. Return only: Yes/No",
"mark_distribution": "If sentence construction is properly formed and pronunciation is acceptable where auditable, give full marks: 10. If either dimension materially fails, give 0. No partial marks. Return only score:",
"detail": "Separate genuine speech problems from ASR errors; pay special attention to amounts, dates, fees, phone numbers, and security instructions."
},
"q_3_7": {
"evaluator_selection_option": "Did the agent communicate at a clear, controlled, and comprehensible pace and pitch? This BRAC Bank version is not a fixed-WPM criterion. Use original audio and speaker timing where available. Return Yes when the agent's speaking rate, pauses, pitch, and emphasis allow a reasonable customer to understand the information, especially amounts, fees, percentages, dates, due dates, card/account instructions, security warnings, and next steps. Return No when delivery is materially too fast, too slow, rushed, monotone/inaudible, erratically pitched, or poorly paused so that important information becomes difficult to understand; or when the customer repeatedly asks the agent to slow down/repeat because of pace/pitch and the agent does not correct it. Precomputed WPM may be used as supporting evidence but there is no public BRAC Bank WPM threshold in this prompt. Do not import Robi's former 350-WPM rule. If audio is unavailable, use timing/repair evidence where available and return No only when a material pace/pitch comprehension problem is clearly evidenced; otherwise return Yes under the current fallback. Never return N/A. Return only: Yes/No",
"mark_distribution": "If pace and pitch are materially clear and comprehensible, give full marks: 10. If a material pace/pitch clarity failure is evidenced, give 0. Never use N/A and never use partial marks. Return only score:",
"detail": "State the audio/timing/comprehension evidence. If WPM is available, report it only as supporting evidence, not as an automatic threshold."
},
"q_3_8": {
"evaluator_selection_option": "Did the agent avoid unexplained internal banking jargon? Return Yes when customer-friendly language is used or a necessary technical term is explained simply. Return No when the agent exposes internal system names, internal queue/disposition codes, underwriting/internal risk codes, CRM/platform names, operations abbreviations, card-processing/acquiring jargon, or other internal terminology without explanation and it could confuse the customer. Do not fail for standard customer-facing terms such as ATM, debit card, credit card, OTP, PIN, NPSB, BEFTN, RTGS, SWIFT, DPS, FDR, eKYC, or ASTHA when used appropriately and explained if the customer appears unfamiliar. Return only: Yes/No",
"mark_distribution": "If no unexplained internal jargon was used, give full marks: 2. If unexplained internal jargon was used in a customer-confusing way, give 0. No partial marks. Return only score:",
"detail": "Identify any internal term used and whether it was explained in customer-friendly language."
},
"q_3_9": {
"evaluator_selection_option": "Did the agent personalize the call respectfully without compromising privacy? Return Yes if the agent appropriately uses a respectful form of address such as Sir, Madam, স্যার, ম্যাডাম, or the customer's verified/preferred name when appropriate. Do not require repeated name use and do not encourage unnecessary disclosure of the customer's full name or account/card details. Return No if the agent never uses an appropriate respectful/personalized form of address in a normal call where there was an opportunity, or uses the wrong/inappropriate form after correction. If the customer clearly requests no honorific/name use, respect that preference and return Yes when the agent complies. Return only: Yes/No",
"mark_distribution": "If the agent uses an appropriate respectful/personalized address without a privacy issue, give full marks: 2. If not, give 0. No partial marks. Return only score:",
"detail": "State the respectful/personalized address used and whether it was privacy-safe and contextually appropriate."
},
"q_3_10": {
"evaluator_selection_option": "Did the agent follow the correct hold and/or transfer protocol when a hold or transfer occurred? If no hold or transfer occurred and none was required, return N/A. When a hold occurs, use supplied BRAC Bank SOP where available. In the absence of an exact internal script, return Yes when the agent explains the reason, seeks/acknowledges permission where practical, gives a reasonable expectation, returns to the customer professionally, thanks the customer after waiting, and does not leave unexplained excessive silence. When a transfer occurs, return Yes when the agent explains why/where the call is being transferred, avoids unnecessary bouncing, protects customer information, and provides a warm/appropriate handoff where supported. Return No for unexplained hold, abandoning the customer, avoidable repeated transfer, wrong/irrelevant transfer, failure to return, or a material privacy/security failure during handoff. Return only: Yes/No/N/A",
"mark_distribution": "If no hold/transfer occurred, give 0 and mark N/A, excluding 4 marks from the applicable denominator. If a hold/transfer occurred and protocol was properly followed, give full marks: 4. If protocol materially failed, give 0. No partial marks. Return only score:",
"detail": "State whether a hold/transfer occurred and the specific permission, reason, wait, return, and handoff evidence."
},
"q_4_1": {
"evaluator_selection_option": "Did the agent check comprehension or offer further assistance before closure, or was comprehension otherwise explicitly confirmed? Return Yes when the agent asks whether the customer understood, needs more information, or needs further assistance. Also return Yes when the customer voluntarily and unambiguously confirms understanding/resolution and clearly initiates closure; do not require a redundant agent question. If the transcript ends without any agent closing statement and no contrary disconnect metadata exists, apply the current-phase customer-disconnect assumption and return Yes because the agent had no normal closing opportunity. Return No when the agent had a normal opportunity, the customer did not explicitly confirm understanding/resolution, and the agent completed a closing without a comprehension/further-assistance check. Return only: Yes/No",
"mark_distribution": "If an agent comprehension/further-assistance check is present, the customer explicitly confirms understanding/resolution and initiates closure, or the current disconnect assumption applies, give full marks: 1. Otherwise give 0. No partial marks. Return only score:",
"detail": "State whether the mark came from an agent check, explicit customer confirmation, or the disconnect assumption."
},
"q_4_2": {
"evaluator_selection_option": "Did the agent close appropriately, politely, clearly, and professionally? Exact BRAC Bank script wording is not required unless a supplied internal SOP explicitly makes it mandatory. Accept natural professional variations such as thanking the customer, offering a courteous goodbye, wishing the customer well, or an appropriate salam. Return No if the agent had an opportunity but the closing was rude, abrupt, careless, incomplete, confusing, or missing. If no agent closing statement appears at the end and no contrary disconnect metadata exists, apply the current-phase customer-disconnect assumption and return Yes. Return only: Yes/No",
"mark_distribution": "If a professional closing variation is present, or the customer-disconnect assumption applies, give full marks: 1. If the agent had an opportunity but closing was unprofessional or missing, give 0. No partial marks. Return only score:",
"detail": "Judge professionalism, not exact script matching. State whether a disconnect assumption was used."
},
"q_5_1": {
"evaluator_selection_option": "Did the agent capture the customer's issue, action taken, and disposition correctly and completely in CRM/wrap-up? Return N/A unless BRAC Bank CRM, case, ticket, complaint, disposition, or wrap-up metadata is supplied. Do not infer CRM capture from transcript alone. When reliable metadata is supplied, return Yes only when the selected category/disposition and key notes accurately match the customer's actual issue, relevant product/channel, action taken, escalation/complaint status, and promised next step where required. Return No when the category is wrong, material notes are missing/mismatched, action/disposition is false, or an urgent fraud/security issue is incorrectly coded in a way that could affect handling. Return only: Yes/No/N/A",
"mark_distribution": "Without BRAC Bank CRM/case/wrap-up metadata, give 0 and mark N/A, excluding 5 marks from the applicable denominator. If auditable and correct, give full marks: 5. If auditable and materially incorrect/incomplete, give 0. No partial marks. Return only score:",
"detail": "State whether CRM metadata was available and whether the case category, notes, action, and disposition matched the call."
}
}