1. Introduction


1.1 Problem Statement

Digital photographs provide powerful and impactful evidence for law enforcement and the courts. However, advanced photographic manipulation techniques and capabilities have elevated concerns about integrity and provenance. Emergency responders need a proactive solution: a smartphone camera app and verification tool that work together to prove image integrity and provenance.

  • Image integrity — whether the image has been altered after being generated
  • Image provenance — proof concerning the origin of an image and how it was created (e.g., when an image was taken, what device was used to capture the image, and where the image was taken)

USE CASE: Evidence Provenance

Investigators are responding to the scene of a forced entry. Using an app on their officially issued mobile devices, they capture images of the scene. These images are entered into evidence in both their investigation and the ensuing court case against an accused perpetrator. Anyone who receives the images can run the verification tool to establish where and when the images were taken, the vendor and model of the smartphone, and whether the images were altered. Because law enforcement can provide proof of both the provenance and integrity of the images taken at the crime scene, their case against the accused is strengthened against accusations of image tampering and forging.

1.2 Background

Protecting Image Integrity

Image tampering is a problem for the courts. Increasingly robust software allows people to make subtle changes to photos that are difficult to detect (e.g., change faces, add a gun, or remove tattoos). There are many ongoing efforts to solve this problem from the forensic end of this problem. For example, DARPA is working on challenges that address this problem from the viewpoint of inspecting images (i.e., pixels, compressed images)[1]. These algorithms do the equivalent of looking at an image to find tampering. This is a reactive solution. Our challenge contest seeks a proactive solution instead.

Early attempts at a proactive solution met with resistance. See for example the Wikipedia discussion of digital watermarking cameras (https://en.wikipedia.org/wiki/Digital_watermarking/). Solutions that alter images within the camera may be viewed as confidence reducing alterations. Law enforcement and the courts must be able to understand the solution before they will trust and use it.

Naive solutions involve the use of cryptographic hashing to provide proof of image integrity. However, the promulgation of these hash artifacts requires an infrastructure to ensure their proper use.

Providing Image Provenance

Digital evidence collection should be a rigorous as its analog counterpart. Consumers expect large drops in quality when moving from DSLR cameras to compact cameras to phone cameras, but in reality, the quality drops are small[2]. Digital smartphones are commonplace multi-purpose tools that public safety can leverage to bolster confidence of photographic evidence as authentic and trustworthy. In this context, image provenance means being able to prove certain characteristics concerning the source of a given image. This includes, but it not limited to:

  • The physical mobile device used to capture the image, represented either as a MAC address, IMEI, or another unique device identifier
  • The physical location (GPS, connected cell tower, etc.) of the device when the image was captured
  • The time at which the image was captured
  • The department’s identifying information (e.g., department name, case number)

1.3 Objectives

The goal of this contest is to build a system that provides Proactive Image Protection (PIP) for a photo taken by a mobile device. This system will be referred to as the PIP solution or the solution. A photo that adheres to the following requirements will be referred to as a PIP image:

  • Image integrity — whether the image has been altered after being generated
  • Image provenance — proof concerning the origin of an image and how it was created (e.g., when an image was taken, what device was used to capture the image, and where the image was taken)

There are three primary deliverables involved in answering this contest:

  • A solution pitch with an overview of the proposed approach
  • A mobile app that generates PIP images
  • A verification tool that examines images and proves PIP

Please reference section 3. Expected Deliverables from Participants for further details about these deliverables.

[1] https://www.darpa.mil/program/media-forensics

[2] https://www.its.bldrdoc.gov/publications/2820.aspx and https://www.its.bldrdoc.gov/publications/3172.aspx

2. Evaluation criteria

Participants must adhere to the basic application requirements listed below. Failure to do so may result in non-grading of the application.


Criteria #0: Basic Requirements

Rating: Pass/Fail

  • Participant submission meets the parameters specified in section 3.1 Deliverable #1: Solution Pitch (below).

Criteria #1: PIP Pitch

Rating (20/100)

These criteria assess whether participants understand the challenge. Using the PIP pitch described in deliverable #1, a public safety official will assess whether the PIP solution:

  • Can be understood
  • Is proactive
  • Seems plausible and inspires confidence
  • The PIP solution cannot store images on a system external to the mobile device.
  • Does not modify the images in a way that may be perceived as corruption or distortion
  • Public Safety can afford any required infrastructure

Criteria #2: Cybersecurity

Rating (20/100)

These criteria assess core functionality of the PIP solution.

  • The PIP solution will operate correctly when mobile access is unavailable, though operating without network may affect the confidence of the provenance of the image (airplane mode)
  • Does the app prompt the user to choose a camera app to take the picture (Pass/Fail)?

Criteria #3: App & Verification Tool User Interface

Rating (20/100)

These criteria assess the mobile app and verification tool.

  • The judges can use the app to take a photograph
  • The judges can copy the PIP images from the smartphone to a laptop
  • Participants demonstrate verification failure (e.g., take a PIP photo, alter it with a hex editor, show that the image fails the verification check)
  • The images generated by the app are standard JPEG files (e.g., jpeg or jpg extension, viewable by image viewers, image editors, and word processors)

Criteria #4: Advanced Cybersecurity

Rating (20/100)

These criteria provide an in-depth assessment of the PIP solution’s cybersecurity, performed by SMEs.

  • The PIP solution is technically sound
  • The PIP solution does not raise concerns as a forensic evidence tool
  • Supporting metadata generated by the PIP solution cannot be used to re-create the original image
  • Technically sound strategies are used to create optional data (e.g., accuracy of date and time, accuracy of geolocation)
  • PIP images that are altered fail their verification check. When the judges run the verification tool, the PIP information on app and verification tool match
    • when an image was taken
    • where the image was taken
    • what device was used to capture the image
    • whether the image was modified
    • which department took this photograph and for what purpose
  • And, optionally
    • how accurate is the date and time?
    • how accurate is geolocation?
  • Any supporting information that is crucial to the PIP solution (other than the actual images) should be protected if they stay resident on the device.
  • If the PIP solution relies on network communication, it must implement Transport Layer Security (TLS) properly
  • The app adheres to the subset of the NIAP Protection Profile for Application Software[1] required for this competition.
  • The app avoids the OWASP Mobile Top 10 security vulnerabilities[2]

[1] https://www.niap-ccevs.org/Profile/Info.cfm?PPID=394&id=394

[2] https://www.owasp.org/index.php/Mobile_Top_10_2016-Top_10

Criteria #5: User Experience

Rating (20/100)

These criteria provide an in-depth assessment of the user experience, performed by SMEs.

  • The app installs and runs on at least one of the following devices: iPhone XS Max, Sonim XP8, and Motorola LEX L11[1]
  • The app adheres to consumer expectations around a camera app (e.g., JPEG/JPG images, point-and-click, images and data easily transferred to and used by other devices and software)
  • The workflow is easy to use (e.g., will minimize user error)
  • The app provides excellent quality of experience (QoE)
  • The PIP solution will scale to thousands of users and devices
  • The PIP solution provides good image quality
  • The PIP solution pitch describes a unique and innovative strategy that may appeal to departments that are dissatisfied with other solutions (e.g., due to unforeseen policy concerns)
  • Satisfies multiple public safety use cases (e.g., law enforcement, EMS)
  • Verification of PIP images is accessible to the greatest number of users at minimal cost to them

[1] https://www.firstnet.com/marketing/index.html#/marketing/index/devices/phones


Review the How to Participate instructions in section 3 of this document to ensure that you are prepared for the in-person Regional Codeathon or Online Contest. The following deliverables will need to be included with the submission:

3.1 Deliverable #1: Solution Pitch

Participants must describe their overall solution to the challenge statement. The solution pitch must contain the following:

  • A high-level description of the PIP solution. This description should highlight the technical model, workflow, and/or algorithms used to create PIP images as well as how PIP verification integrates into this workflow.
  • Description of how the solution will enable users in heterogeneous environments to verify PIP images (e.g., prosecution and defense teams).
  • Specific information on the following security concerns:
    • How the app will protect any sensitive data generated by the app at rest (stored in permanent storage) on the device
    • How the app will protect any sensitive data generated by the app that is transmitted off the device
    • How the participants approach code quality and best practices for design
    • How the verification will protect any sensitive data
    • How the verification will prove PIP

The solution pitch can either be:

  • A written document, not to exceed two pages
  • A slide presentation, not to exceed 10 slides
  • A narrated video, not to exceed 3-5 min 

3.2 Deliverable #1: Mobile App

Participants must create a working camera app with the following functional characteristics:

  • The app must run on at least one of the following devices: iPhone XS Max, Sonim XP8, and Motorola LEX L11
  • The PIP image must be a standard JPEG file
  • The algorithm(s) used to establish PIP must be transparent (that is, they must be describable to activity judges).
  • The PIP solution must not make images available to a third party
  • Metadata that is part of the PIP solution cannot be used to re-create the original image
  • The app must use strong cryptographic methods including, if using network communications, network transport security
  • The user interface must adhere to consumers’ expectations around camera apps (e.g., easy to use and images can be viewed with the smartphone’s default image viewer)
  • Competition judges can retrieve images from the smartphone via USB or via a network transfer
  • The PIP images may be viewed with any JPEG compliant software
  • The app must not use other camera apps to take the pictures (this because, we cannot be sure to the security standing of the other camera apps)

The following characteristics are desirable but not required:

  • The PIP solution establishes geolocation with accuracy (e.g., plus or minus 100 feet vs. 0.5 miles)
  • The PIP solution establishes calibrated date and time, with due diligence proving accuracy (see ANSI X9.95)
  • The PIP solution includes advanced planning for forensic image post-processing that is beyond the scope of this challenge (e.g., forensic image enhancement software takes a PIP image, applies auto-contrast corrections, redacts faces, and saves the modified image as a new PIP image that specifies the processing chain)

3.3 Deliverable #2: Verification Tool

Participants must create a verification tool (e.g., software package or website). The verification tool does not need to be publicly available when the PIP solution is submitted. The verification tool must have the following functional characteristics:

  • When given a PIP image, provides PIP information
  • Otherwise, indicates that PIP cannot be established (e.g., PIP image has been modified, JPEG file is not a PIP image)
  • The verification tool should be able to run on a laptop running Windows 10 or Ubuntu 18.04
  • Is easily operated by naïve users

3.4 Additional Deliverables

  • A completed submission form through techtoprotectchallenge.org
  • A 3-minute narrated PowerPoint file or 3-minute narrated video with the following sections (this should be an updated version of the video or complementary to the deliverable for 3.1):
    • A summary of the submission.
    • Basic information about the participant or the participant’s team.
    • Specific requirements of the contest that are being addressed by the participants.
    • Two to four screenshots of the solutions or prototype.
    • Overview of key functions included in the submission.
  • Any available files or links to the developed solution or prototype.
  • Any additional files based on the contest description.
Tech To Protect
Rules & Guidelines