We are working on this.

Hope this help you out.

Cleveland Municipal Court
Ohio Open Records Law Office
Level Two Court Tower
1200 Ontario Stree
Cleveland, OH 44113

November 25, 2019

To Whom It May Concern:

Pursuant to the Ohio Open Records Law, I hereby request the following records:

Records and materials related to the solicitation, acquisition, evaluation, trial, pilot, and use of the Public Safety Assessment (PSA), created by the Laura and John Arnold Foundation (now known as Arnold Ventures) or any equivalent risk assessment system used by your agency or associated criminal justice offices.

This "software" or services may be provided by the Arnold Foundation, though this request is applicable to this, as well as any other company providing predictive risk assessment tools related to the criminal justice system of this jurisdiction or under consideration or contract with this agency for similar services.

Materials responsive materials include but are not limited to:

1. ALGORITHM or CODE
Any algorithm or code developed for implementation of or as part of this "software" or program or any related system that conducts calculations or decision making. This includes any algorithm or software developed by or with, given to, used by, purchased or licensed to this agency for implementation of the above referenced program, as well as any algorithm or source code in use by this program or by which this system conducts calculations or decision making. This would include original source code, any compiled binaries (also known as executables), specification documents, spreadsheets, program scripts, and other digital materials used to calculate any data in the above program.

2. INPUT DATA
Any input training data for the "software" or related machine learning algorithms or programs.

For the aspects of the software that require an input (for example, to compute a value), please provide:
• a copy of the five most recent sets of data that were used for input, as well as
• the five most recent outputs of the software

in whatever their native format is.

If these inputs or outputs include exempt information, please provide all material save for specific information that is specifically exempted by law. Please also provide a description of input and output data fields that will aid in understanding the type of information that is submitted to the software and that is produced by the software.

3. AUDITS, REPORTS, and VALIDATION STUDIES
A copy of all reports concerning or mentioning the "software," including audit records, annual reports that mention the use of the "software," reports to legislative bodies, misuse reports, reports to oversight bodies.

This also includes all impact and validation studies. Please provide a copy of any validation studies conducted with respect to the program or with respect to any software or algorithms used in connection with the program. Please also provide a copy of any records pertaining to inquiries for the need of validation studies or discussion of potential or actual validation studies. A “validation study” in this context is any study designed to assess how accurate the program is in predicting what it is supposed to predict, used to assess whether the program may err in the direction of either under- or overestimating likelihoods or predicted outcomes, or intended to evaluate whether the software does or will produce any results that are biased or unexpected in any way.

4. AGREEMENTS and CONTRACTS
Any and all agreements related to the acquisition and use of this "software," algorithm, or program.

This includes (but is not limited to):
• all contracts (as well as associated amendments, attachments, and exhibits),
• data sharing agreements
• insurance agreement
• intergovernmental services agreements
• licensing agreement
• memorandums of understanding, and
• nondisclosure agreements

regarding or relevant to this software or program.

Please provide a copy of any informal agreements, insurance agreements, liability waivers, and warranties that reference this technology or otherwise guide its use or the relationship between this agency, the provider or developer of the "software," or any other partner or entity that may use or access it.

5. BIDDING and PROCUREMENT
A copy of any available or relevant Requests for Expressions of Interest, Requests For Proposal (or equivalent calls for bids), responses to Requests for Expressions of Interest, letters of interest, responses to Requests for Proposal, sole source or limited source justification and approval documentation, documentation of selection, and other materials generated in the consideration and selection of the technology in question.

6. FINANCIAL and FUNDING MATERIALS
Any records related to the financing or funding of this "software," including a copy of any related funding opportunity announcements, grant applications, grantor status/progress reports, purchase orders, invoices, and other memoranda and documentation related to the payment or cost (or lack thereof) of the "software," related technology, personnel, equipment, or other elements concerning this "software" or program.

7. INSTRUCTIONAL MATERIALS and TRAINING
All instructional materials, presentations, and presentation materials (including recorded video and audio, PowerPoint files, prepared remarks, and slides formats), and other guidance on the use of "the software."

This includes any notes taken during meetings that discussed the use of the software, any explanations (whether generated internally or externally) of how the software works, and any other document that has been used to help explain the use of "the software" to any party, including internal documentation, public relations materials, and executive summaries.

This also includes training material governing the use, sharing, or access to the "software" or any data related to or collected by the face recognition software/technology, including the legal standard that is required before using the technology.

This also includes any description of input and output data fields that will aid in understanding the type of information that is submitted to the software, and that is produced by the software.

8. PRIVACY ASSESSMENTS
A copy of any data retention guidelines or policies, data security policies, data security specifications, privacy impact assessments, security audits, or other materials evaluating or guiding the security of the "software" or the privacy of the data involved

9. USE POLICIES
Please provide a copy of any policy directives, guidance documents, memoranda, training materials, or similar records governing the use and function of this technology for immigration, law enforcement, or any purpose, including all those related to data retention, permissible and impermissible use, and security standards. This would include materials that describe the application, function, and use of the "software," including advertisements, emails, handouts, usage policies, PowerPoint presentations, specification documents, or standard operating procedures.

10. COMMUNICATION and MARKETING MATERIALS
All communications or marketing materials relevant to or mentioning this "software" or program.

This includes any correspondence between any staff at this agency with any other organization or vendor relevant to the use of the aforementioned technology or program, including records related to meetings or follow-up actions with any vendors, companies, or other private entities marketing this technology for immigration, intelligence, law enforcement, or any use. This also includes correspondence in the form of emails, email attachments, paper letters, text message, and any other form of correspondence, as well as all marketing materials - unsolicited, requested, or otherwise - acquired from vendors of this technology.

____

If your office has questions about this request, please feel free to direct them to the address associated with this request or call the MuckRock office at 617-299-1832.

The requested documents will be made available to the general public, and this request is not being made for commercial purposes.

In the event that there are fees, I would be grateful if you would inform me of the total charges in advance of fulfilling my request. I would prefer the request filled electronically, by e-mail attachment if available or CD-ROM if not.

Thank you in advance for your anticipated cooperation in this matter. I look forward to receiving your response to this request within 10 business days.

Sincerely,

Beryl Lipton

Filed via MuckRock.com
E-mail (Preferred): requests@muckrock.com<mailto:requests@muckrock.com>
Upload documents directly: https://www.muckrock.com/
Is this email coming to the wrong contact? Something else wrong? Use the above link to let us know.

For mailed responses, please address (see note):
MuckRock News
DEPT MR 83730
411A Highland Ave
Somerville, MA 02144-2516

PLEASE NOTE: This request was filed by a MuckRock staff reporter. Also note that improperly addressed (i.e., with the requester's name rather than "MuckRock News" and the department number) requests might be returned as undeliverable.
[http://email.requests.muckrock.com/o/eJwNykESgyAMAMDXlCOTQIL2wKW2_UZHQ6iOMoyo_2_3vCkyB6NlXLbPkiICMTowc-wmGPsQ-kyaJ1HoQVJSIk7Bu0zJLNEB3hEdI1NwaNE-B-rQMz3AE73e_kbQdL_0OA9bLllbldVKLabFYdO21jzUq53H_0mROtvavj-MlSsm]

This might be helpful to you as well.

http://clevelandmunicipalcourt.org/contact-us/history/archive/judicial-division/pretrial-services/public-safety-assessment-(psa)-training

It took some doing, but I’m glad you have it.

Ed Ferenc