What do I need to do if a Vendor's products or services use Artificial Intelligence (AI)?

It is important to understand how a vendor's services/products may use Artificial intelligence (AI) capabilities to ensure that use aligns with UC's policies, advisories, and guidelines on AI. AI functionality in Vendor services/products must be evaluated for security, privacy, and general AI risks.

Regarding the Vendor Security Assessment process and AI, the following are some key questions the Requester should be prepared to answer in coordination with the Vendor.

ISO will ask you to provide answers to these types of questions when we initially triage your VSA request:

  1. How is AI being used in the vendor's service/product? Please provide a detailed response. Technical descriptions are encouraged. 
    • Be prepared to provide links or copies of the vendor's documentation describing how AI is being used and relay any vendor's responses to these questions.
    • For example, ISO will need to understand what types of AI are utilized (e.g., generative AI, machine learning) and how they interact with UC Institutional Information or IT Resources to assess security impacts. 
  2. Is customer data (UC) used to train the Vendor's AI models or mechanisms?
  3. What types of data are processed and/or collected by the Vendor's AI services/products, and for what purpose?
  4. If UC data is collected by the Vendor's AI services/products, is it anonymized?
  5. Does the Vendor's use of AI involve highly consequential automated decision-making on behalf of UC? If so, please describe.
    • Examples include, but are not limited to:
      • Legal analysis or advice
      • Recruitment, personnel, or disciplinary decision-making
      • Seeking to replace work currently done by represented employees
      • Security tools using facial recognition
      • Grading or assessment of student work
  6. Have you engaged the UC Berkeley Privacy Office to evaluate the privacy impacts of the Vendor's AI services/products?
    • If you have not already, we recommend you reach out to begin the process as a privacy impact analysis on AI is needed in most cases. 

Additional AI Resources