Transparency

Rules & Examples Library

The evaluator uses deterministic rules stored in this repository. Update these JSON files to add or edit guardrails.

Rules (11)

phi-or-credentials-red

RED

Protected health data or credentials must stay offline

privacysecurityphi

Conditions

{
  "dataType": [
    "phi",
    "credentials"
  ]
}

Rationale

  • PHI/credentials cannot be sent to third-party or cloud AI tools
  • Risk of HIPAA/FERPA/contract breaches
  • No downstream audit controls once uploaded

Safer workflow

  • Strip identifiers or secrets before any tooling
  • Work in an offline or enclave environment
  • Consult security/IRB before any external processing

human-identifiable-irb-pending

RED

Identifiable human data with pending IRB

privacyirbphi

Conditions

{
  "dataType": [
    "human_subjects",
    "phi"
  ],
  "deidentification": [
    "identifiable",
    "not_sure_deid"
  ],
  "context": "irb_pending"
}

Rationale

  • Identifiable human data cannot be shared before IRB approval
  • Cloud AI tools expand the data exposure surface
  • IRB must approve storage/processing locations

Safer workflow

  • Pause AI usage until IRB is approved
  • If possible, fully de-identify data
  • Use secure local analytics under protocol once approved

human-limited-irb-approved-enterprise

YELLOW

Limited human data with IRB approval and enterprise tool

irbprivacy

Conditions

{
  "dataType": "human_subjects",
  "deidentification": [
    "limited",
    "deidentified"
  ],
  "context": "irb_approved",
  "destination": "copilot"
}

Rationale

  • IRB-approved with limited identifiers still requires caution
  • Enterprise Copilot provides better contractual controls
  • Verify data handling matches the approved protocol

Safer workflow

  • Remove direct identifiers before upload
  • Reference protocol number in your notes
  • Limit prompts to necessary excerpts only

copyright-no-permission-consumer

RED

Copyrighted content without permission to consumer chatbot

copyright

Conditions

{
  "dataType": "copyrighted",
  "copyrightPermission": [
    "no_permission",
    "fair_use_uncertain"
  ],
  "destination": "consumer_chatbot"
}

Rationale

  • Consumer chatbots may reuse or retain uploaded copyrighted works
  • No license or uncertain fair use increases takedown risk
  • Terms of service often allow model training on inputs

Safer workflow

  • Seek permission or licensed source
  • Use offline summarizers or note-taking tools
  • Share only short excerpts that meet fair use guidance

copyright-no-permission-internal

YELLOW

Copyrighted content without permission, internal use

copyright

Conditions

{
  "dataType": "copyrighted",
  "copyrightPermission": [
    "no_permission",
    "fair_use_uncertain",
    "not_sure_permission"
  ],
  "destination": [
    "copilot",
    "local"
  ]
}

Rationale

  • Internal tools reduce distribution risk but license is uncertain
  • Only minimal necessary excerpts should be used
  • Document reasoning for fair use

Safer workflow

  • Confirm license or obtain permission
  • Use small excerpts, avoid whole works
  • Record justification in project notes

draft-to-consumer-yellow

YELLOW

Unpublished draft to consumer or unknown vendor

confidentiality

Conditions

{
  "dataType": "draft",
  "destination": [
    "consumer_chatbot",
    "third_party"
  ]
}

Rationale

  • Drafts may contain unpublished findings
  • Consumer tools can leak or train on uploads
  • Check publisher or sponsor restrictions before sharing

Safer workflow

  • Share only non-sensitive sections
  • Strip figures/data before using AI
  • Use enterprise-approved environments

draft-enterprise-green

GREEN

Unpublished draft in enterprise tool

confidentiality

Conditions

{
  "dataType": "draft",
  "destination": "copilot"
}

Rationale

  • Enterprise tools keep data within institutional boundary
  • No legal restrictions noted for drafts
  • Lower risk when scope is editing or outlining

Safer workflow

  • Avoid uploading embargoed figures
  • Use short excerpts for editing
  • Log usage in project notes

public-green

GREEN

Public or non-sensitive data

low_risk

Conditions

{
  "dataType": "public",
  "destination": [
    "copilot",
    "local"
  ]
}

Rationale

  • Information is already public or non-sensitive
  • Enterprise or local tools keep usage traceable
  • No additional approvals required

Safer workflow

  • Avoid adding any identifiers
  • Keep prompts general
  • Document outputs in notes if used in reports

proprietary-third-party

RED

Proprietary or NDA data to consumer/unknown vendor

ndacontract

Conditions

{
  "dataType": "proprietary",
  "destination": [
    "consumer_chatbot",
    "third_party"
  ]
}

Rationale

  • NDA/contract data cannot be shared with unvetted vendors
  • Terms may allow model training on uploaded content
  • Potential breach of confidentiality obligations

Safer workflow

  • Confirm contract allows AI tooling
  • Remove partner identifiers before any upload
  • Prefer enterprise or offline analysis

student-records-ferpa

YELLOW

Student record data

ferpaprivacy

Conditions

{
  "dataType": "student_records"
}

Rationale

  • Student data is protected (FERPA-like)
  • Only approved tools with institutional agreements should be used
  • Consider de-identifying grades or IDs

Safer workflow

  • Remove names and IDs before prompts
  • Use aggregate statistics when possible
  • Log use in academic records handling notes

not-sure-default-yellow

YELLOW

Not sure requires verification

uncertainty

Conditions

{
  "destination": "not_sure_destination",
  "context": [
    "not_sure_context",
    "irb_pending"
  ]
}

Rationale

  • Unclear destination or approvals warrants caution
  • Verifying tooling and approvals prevents accidental disclosure

Safer workflow

  • Confirm approved vendor or contract
  • Escalate to IRB/privacy officer if human data
  • Use synthetic or sample data until confirmed

Examples (10)

ex-phi-identifiable

RED

PHI identifiable going to Copilot

Answers

{
  "dataType": "phi",
  "destination": "copilot",
  "intent": "analyze",
  "context": "irb_pending",
  "deidentification": "identifiable"
}

Notes: PHI stays red regardless of destination

ex-human-limited-irb-approved

YELLOW

Human subjects limited data, IRB approved, Copilot

Answers

{
  "dataType": "human_subjects",
  "destination": "copilot",
  "intent": "summarize",
  "context": "irb_approved",
  "deidentification": "limited"
}

ex-copyright-consumer

RED

Copyrighted article to consumer chatbot without permission

Answers

{
  "dataType": "copyrighted",
  "destination": "consumer_chatbot",
  "intent": "summarize",
  "context": "no_irb",
  "copyrightPermission": "no_permission"
}

ex-copyright-internal

YELLOW

Copyrighted excerpt internally

Answers

{
  "dataType": "copyrighted",
  "destination": "copilot",
  "intent": "edit",
  "context": "no_irb",
  "copyrightPermission": "fair_use_uncertain"
}

ex-public-green

GREEN

Public content in enterprise tool

Answers

{
  "dataType": "public",
  "destination": "copilot",
  "intent": "brainstorm",
  "context": "no_irb"
}

ex-draft-consumer

YELLOW

Unpublished draft sent to consumer chatbot

Answers

{
  "dataType": "draft",
  "destination": "consumer_chatbot",
  "intent": "edit",
  "context": "no_irb"
}

ex-draft-enterprise

GREEN

Draft in enterprise Copilot

Answers

{
  "dataType": "draft",
  "destination": "copilot",
  "intent": "summarize",
  "context": "no_irb"
}

ex-proprietary-thirdparty

RED

NDA data to unknown vendor

Answers

{
  "dataType": "proprietary",
  "destination": "third_party",
  "intent": "analyze",
  "context": "contract_restrictions"
}

ex-student-records

YELLOW

Student records with Copilot

Answers

{
  "dataType": "student_records",
  "destination": "copilot",
  "intent": "summarize",
  "context": "no_irb"
}

ex-not-sure

YELLOW

Not sure destination and context

Answers

{
  "dataType": "public",
  "destination": "not_sure_destination",
  "intent": "brainstorm",
  "context": "not_sure_context"
}