カスタマーサポート品質保証：18項目チェックリスト

クイック回答

transcriptを開く前にpurpose/sampleを定義。observable evidenceだけを採点しnot scorableを許可、critical failureとweighted coaching itemを分離、source/reviewer/decision trailを保持。同一caseでreviewer calibration。QA、CSAT、speedを別signalとして分析しcorrective workのcustomer-visible outcomeを検証します。

score evidenceinteraction、system、policy、final customer-visible stateで実際に起きたこと。

abstentionmissing contextはzeroではなくnot scorable理由を記録。

close looprecurring failureをowned actionにしてeffectをverify。

reviewerが説明可能なscoring method

scoring前に定義

criterion、evidence rule、weight、critical-failure rule、scorable population、periodをversion化。low scoreを見てrubricを後調整しない。

4 outcome

pass、fail、partial、not scorableにcriterion-specific anchor。missing segmentはobserved failureではない。confidence/evidence link保持。

gateとcoachingを分離

security、privacy、unauthorized financial action、fabricationはhigh weighted scoreでもmandatory fail。coaching itemでunsafe interactionをacceptableにしない。

minimum review record

review_id · purpose · population · sampling_method · conversation_id · channel · locale · human_or_ai · rubric_version · criterion_id · outcome · evidence_pointer · reviewer · confidence · critical_failure · disagreement · calibration_set · appeal · corrective_action · owner · due_date · effectiveness_check

18-point customer support QA checklist

各controlはinspect対象、score evidence、fail conditionを示します。serviceに合わせanchorを調整し、全項目をequal-weight checkboxにしないでください。

scopeとsamplingが妥当

採点前にchannel、queue、language、product、customer tier、human/AI participation、review window、eligibility、sampling methodを確定。escalationだけのconvenience sampleはteam qualityを代表しない。

score evidence: 再現可能なpopulation/ticket ID、exclusion、random/stratified selection、reviewer、comparison dimension。
fail condition: cherry-pick、duplicate隠蔽、defined population外への一般化はfail。

monitoringの透明性とdata minimization

quality purposeを文書化し必要時worker/customerへ通知。reviewer/modelには必要fieldのみ。credential、payment、health、secret、restricted investigationを分離。

score evidence: notice/purpose、必要なbasis、field inventory、access role、retention/deletion、redaction test。
fail condition: covert/excessive monitoring、reviewなしのpurpose変更、不要sensitive narrativeのAI送信はfail。

interaction contextを完全に保持

customer message、prior thread、channel、timestamp、attachment、automation event、ownership change、final stateを確認。一文だけをwhole interactionとして採点しない。

score evidence: immutable transcript/recording reference、timeline、actor type、edit、missing segment、channel constraint、final state。
fail condition: context欠落なのにintent、fault、response time、resolutionを推定したらfail。not scorableにする。

customer needとdesired outcomeを特定

customer task、affected object、expected/observed result、urgency、requested outcomeをevidence-linkedに記述。stated requestとunderlying issueを分離。

score evidence: quoted/timestamped evidence、normalized issue、affected object、desired outcome、uncertainty、competing interpretation。
fail condition: goal創作、multiple issueのcollapse、sentimentをproblem definition扱いはfail。

fact/instructionが正確で根拠あり

response時点のproduct behavior、account state、policy/knowledge version、tool resultを確認。verified fact、hypothesis、unknownを分離。

score evidence: source/version、account/tool evidence、retrieval time、policy、calculation input、confidence、correction trail。
fail condition: fabricated feature、stale step、unsupported cause、invented tool result、evidenceなしの断定はfail。

consequential assumption前にclarify

identity、version、jurisdiction、account state、date、scope、remedyで回答が変わるなら最小限質問。既存dataを再要求しない。

score evidence: decision-changing ambiguity、question、context check、answer、safe temporary path、no-response handling。
fail condition: material assumptionでaction、routing/resolutionに無関係な質問でfrictionはfail。

policy・eligibility・exceptionを正しく適用

customer、event date、region、plan、requestに適用されるpolicy/termを使用。conditionを説明しexceptionをroute。

score evidence: policy ID/version/date、eligibility input、exception path、decision owner、explanation、conflict record。
fail condition: wrong date/region、invented exception、policyをlegal adviceとして提示はfail。

identity・permission・sensitive actionをcontrol

必要levelだけidentity verify、approved channelとrole permissionを使用。refund/delete/credential/irreversible actionは追加authorization。free textでsecret要求禁止。

score evidence: verification、actor/role、permission、preview、approval、idempotency、audit、rollback。
fail condition: over-verification、secret収集、privilege escalation、approvalなし、duplicate action、cross-customer leakはfail。

severity・priority・routingがevidence一致

impact、urgency、affected user、workaround、security/privacy signal、service state、contractで分類。uncertaintyを保持しcurrent routing matrix使用。

score evidence: severity input、priority rule/version、scope、incident link、specialist queue、SLA、override/approver。
fail condition: angry toneだけでseverity上昇、calm reportでincident見逃し、AI confidenceでspecialist review代替はfail。

directでusableなnext stepを先頭に

answer/current state/next actionを冒頭。prerequisiteをcommand前、step順序、owner、required/optionalを明示。

score evidence: first answer、ordered step、prerequisite、owner、observable result、stop condition、alternative。
fail condition: 長いpreamble、order不正、successがobservableでない場合fail。

respectful・specificで非performativeなtone

具体的inconvenience/riskを認めるがfeeling、blame、certainty、intimacyを創作しない。urgency/channelに合わせprofessional/inclusive。

score evidence: impact acknowledgment、neutral ownership、known failureへのapology、no blame、no manipulation。
fail condition: canned empathy、unkeepable promise、argument、stereotype/shameはfail。

clear・accessibleでlocalizationが正確

短文、descriptive link、heading、text alternative、audience用語を使用。wordだけでなくtask/policy meaningを翻訳しlocale date/currency/routeを保持。

score evidence: reading level、glossary、locale reviewer、link purpose、attachment alternative、date/timezone、accessible channel。
fail condition: jargon、eligibility/safetyを変えるmachine translation、inaccessible image/audioだけの情報はfail。

ownershipとhandoffがcontinuous

current owner、destination、reason、evidence package、expected response、fallbackを明示。authorized transfer済み情報をcustomerに再説明させない。

score evidence: from/to owner、time、reason、evidence/permission、acknowledgment、notice、fallback/timer。
fail condition: blind transfer、orphan、circular routing、conflicting promise、unneeded data exposureはfail。

time/status expectationがtruthful

measurableなnext-update time/eventとtimezoneを提示。internal targetとcontractual commitmentを分離。期限前にupdateしassumption変更を説明。

score evidence: SLA/target source、due time/timezone、dependency、state、next update、breach warning、revised owner。
fail condition: soonだけ、authorityなしのguarantee、silent miss、incorrect SLA pauseはfail。

resolutionをoriginal needに対しcomplete verification

agent reply/tool successでなくrequested outcomeまたはagreed alternative達成を確認。customer-visible stateと全issue unitをtest。

score evidence: action result、before/after、customer-visible verification、unresolved list、acceptance evidence、reopen path。
fail condition: backend success=customer success、issue見落とし、propagation/confirmation前closeはfail。

workaround/escalationがsafeでbounded

workaroundの変更、user、duration、side effect、monitoring、reversalを明示。security/privacy/legal/financial/health/safety/irreversibleはapproved specialist route。

score evidence: risk、audience、expiry、side effect、monitoring、rollback、specialist acceptance、stop。
fail condition: control bypass、reviewなしでpermanent、incident隠蔽、authority外unsafe actionはfail。

AI participation・uncertainty・human authorityがvisible

reply、summary、score、tag、actionのautomation由来を識別。consequential useではmodel/version、input、evidence、confidence/abstention、human decision、overrideを保持。

score evidence: actor type、model/prompt/workflow version、source、proposed score、human decision、override、incident/rollback。
fail condition: AI outputをverified evidence扱い、same model self-grade、boundary外consequential actionはfail。

closure・feedback・learningをcontrolled loop化

what changed/remains、reopen/escalate、feedback時期を要約。eligible interactionだけCSAT、response rate/selection biasをQAと分離。recurring failureをowned coaching/content/product/controlへ。

score evidence: closure summary、unresolved、reopen、survey eligibility/time、denominator、action owner、due date、effectiveness check。
fail condition: unresolvedを隠すclosure、survey scoreを全真実扱い、contextなしpunitive use、owner/verificationなしactionはfail。

calibration case：2 reviewerが妥当な理由でdisagree

このhypothetical exampleはmethod説明でOpenMax resultではありません。duplicate subscription chargeの返金依頼。agentはapproved verification、two captured charge確認、one refund submit後「tomorrow」と案内。しかしpayment railはmulti-day estimateでfollow-up owner記録なし。

Reviewer A：mostly resolved。 authorized financial actionは一度だけ完了。identity、evidence、idempotency、tone、directnessはpass。
Reviewer B：expectation fail。 tomorrowはunsupported、rail estimateなし、owner/checkpointなし。time/statusとclosureはfail、recordはpartial。
calibration decision。 両observationを保持しvague averageにしない。successful refund、false timing、missing follow-up、必要なcustomer-visible verificationを記録。
corrective action。 templateにrail-specific estimate、next-check date/ownerを必須化し、training assignmentでなくlater case sampleでcontrol効果をverify。

checklistでなくQA programをoperate

calibration / disagreement

ordinary/edge/high-risk caseでindependent review、criterion decision比較、adjudication、prospective anchor更新、human-human/human-AI agreement monitoring。agreement≠correctness、expert ground truth/appeal保持。

denominator / uncertainty

eligible、sampled、scorable、not scorable、criterion outcome、critical failure、disagreement、appeal、corrective action、verified follow-upを報告。channel/locale/product/human-AIで慎重segmentしsmall group ranking禁止。

findingをowned changeへ

defectをcoaching、knowledge、policy、product、workflow、staffing、accessibility、localization、security、privacy ownerへ。acceptance/due date設定、release後comparable sample、correlationとcausal improvementを分離。

OpenMaxによるsupport QA coordination

OpenMaxはeligible sampling frame、minimized review view、transcript/system evidence、evidence-linked criterion proposal、insufficient context時abstain、critical finding human route、override/appeal、corrective work、follow-up sampleをcoordinate。rubric ownership、worker/privacy、specialist judgment、employment consequence、customer remedy、final acceptanceは人が保持。

1 · framepurpose、population、sample、notice、rubric version

2 · gatherminimized transcript、timeline、policy、tool、outcome evidence

3 · proposecriterion outcome、evidence、uncertainty/abstention

4 · reviewcalibrated human decision、critical route、override、appeal

5 · improveowned action、acceptance evidence、comparable re-sample

privacy・fairness・interpretation boundary

QAをundisclosed surveillanceにしない。necessity/proportionality、notice、worker consultation、minimum access、retentionを事前定義。
voice、accent、writing style、sentimentからprotected trait、emotion、honesty、intentを推定しない。observable behavior/evidenceを評価。
comparable population、denominator、calibration、uncertainty、appealなしにagent/vendor/locale/AIをrankしない。scoreはdecision inputでground truthではない。
CSAT response、satisfaction、QA pass、FRT、resolution、reopen、business outcomeは別measure。適切designなしにcausal story化しない。

情報源、編集方法、制限

OpenMax編集部はIntercom conversation-rating eligibility/reporting definition、NIST AI RMF role/measurement/human oversight/monitoring、UK ICO worker monitoring/call-center example、WCAG 2.2を確認し、original 18-control rubric、evidence anchor、failure condition、calibration caseを作成。2026年9月3日再確認。

範囲注記 vendor definitionは自社product、NIST AI RMFはvoluntary、ICOはUK data protection、WCAGはaccessibility。いずれも本rubricをcertifyせずuniversal employment policy/business outcomeを保証しない。実deploymentでprivacy/legal/labor/security/accessibility reviewが必要。

よくある質問

CSATとQAは同じ？

いいえ。CSATはeligibleかつrespondしたcustomer feedback、QAはsampled evidenceへdefined rubric適用。response denominator、QA sample、scorable populationを分離。

18項目をequal weightにする？

通常no。criterion-specific anchor/weightを事前設定。security/privacy/accuracy/unauthorized actionはgateとしtone scoreで平均化しない。

AIは全conversationをscoreできる？

sufficient permitted evidenceならproposal可能。missing/restricted contextではabstain、evidence/uncertaintyを示しconsequential findingをcalibrated human review、override/appealへ。

何conversationをreview？

universal numberなし。decision、volume、variation、risk、precision、strata、capacity次第。eligible population、selection、sample、not-scorable、limitationを公開。

QA findingをどうservice improvementへ？

recurring failureをcoaching、knowledge、policy、product、workflow、staffing、localization、accessibility、security、privacy ownerへ。acceptance、due date、comparable sampleを定義。