1. Introduction

1. INTRODUCTION

The objective of this challenge is to add innovative mapping capabilities to the LTE Coverage Tool Android application (https://play.google.com/store/apps/details?id=gov.nist.oism.asd.ltecoveragetool). The application, in its current state, uses a commercial Android smartphone (also known as a UE, or User Equipment) to provide an assessment of LTE (Long Term Evolution) network coverage.

1.1 Problem Statement

Emergency responders work every day in environments which present challenges for communications, depending on installed cellular infrastructure which may not cover remote locations, interior/underground settings or areas where resources have been compromised. As public safety agencies increase their reliance on cellular/LTE communication, they have a critical need for evaluating coverage in these diverse environments — a procedure which requires expensive equipment and training beyond the budgets of most agencies or teams. Addressing that problem, the U.S. Department of Homeland Security (DHS) sponsored work in the National Institute of Standards and Technology (NIST), Communications Technology Laboratory, Public Safety Communications Research (PSCR) Division and the National Telecommunications and Information Administration (NTIA), Institute for Telecommunication Sciences (ITS) to develop and use an experimental Android application which demonstrated that measurements from commercial UEs (smartphones) could provide a reliable assessment. NIST PSCR released the experimental application and source code in 2018, and NTIA ITS will publish a technical report on their findings in 2019.

1.2 Objectives

Release of the LTE Coverage Tool application and source code provide agencies with a simple, free tool for making subjective coverage assessments, and a code base for customization or integration with other tools. Other applications are also available at low cost, providing similar capabilities (through proprietary software); some of these can present the data using GPS as an overlay onto a map (e.g., Google Maps or OpenStreetMap); however, no known application provides coverage information with location data for environments lacking GPS coverage. The goal for this challenge is to integrate one or more innovative location mapping elements into the tool which will provide a transformational step forward for emergency responders and will translate directly to saving the lives of emergency responders and citizens. Required output for a successful challenge submission will include a working Android Package (APK file). Applications submitted shall incorporate all functionality from the baseline LTE Coverage Tool with additional capabilities for presenting measurements with associated location data as explained below.

A foundational requirement for the submitted application is that no external equipment may be required. Using only the resources available from a commercial UE, the extended application should support three modes for location tracking and presentation (notional diagrams are presented below). In all modes, the map outputs shall present excellent, good, and poor coverage areas, as determined by the baseline app from Android Reference Signal Received Power (RSRP) measurements.

1.2.1 Mode 1: Heat map using GPS to present data on a map (primarily for outdoor use).

Mode 1 is intended for the outdoor use case, where an agency may need to assess coverage in advance of an event or in preparation for incidents. GPS coverage is assumed for this case. Mode 1 is the starting point for a submission, since it should be relatively easy to overlay coverage quality data from the baseline app as a heat map.

1.2.2 Mode 2: Where no GPS data or building plan is available, represent route as a heat map using internal UE sensors.

For Mode 2, it is assumed that GPS is not available or is intermittent throughout the assessment route. Accessing any internal UE sensors, the application shall generate a conceptual map of a route (walked by a user), such that after completion, the user will be able to discern excellent, good, and poor coverage areas along the route (i.e., within a building).

The application shall not access external hardware for position determination or tracking. Innovative mapping presentations are encouraged!

Evaluation routes will include multiple levels, and the application must display levels/floors in some manner (layers, multiple views, 3D view, etc.). Innovation is encouraged!

1.2.3 Mode 3: Given a building plan/map and planned route, overlay heat map onto route.

Mode 3 expands upon Mode 2, using the same assumptions and requirements and adding the capability for a user to load a building plan (image file) and plot the conceptual map onto the plan. It is expected that the user would click on the starting point and enter course corrections (corrected locations) along the route. Example image files will be made available to participants.

The primary use case for Mode 3 is intended for site inspections or similar scenarios where an emergency responder would have a building plan prior to starting an assessment.

1.3 Resources

The following resources will be made available to participants for the challenge:

  • Source code published on the NIST GitHub (required as a base for all submissions): https://github.com/usnistgov/LTECoverageTool
  • Template for CSV file which will be required for evaluation.
  • Template for building plan (mode 3).

2. Evaluation criteria

Participants must adhere to the basic application requirements listed below. Failure to do so may result in non-grading of the application.

Criteria #0: Basic Requirements

Rating: Pass/Fail

  • Delivery of screenshots of maps from each mode
  • Route maps with descriptions to support map screenshots

Criteria #1: UI/UX Evaluation, Mode 1 Operation

Rating: 20/100

  • Subjective scoring of user interface/user experience based on Android Material Design Basics (https://developer.android.com/design/) and Mode 1 mapping presentation.
  • Rating (20/100)
    • UI/UX (layout, style, components, patterns, visual quality, Mode 1 presentation): 15/100
    • Mode 1 map performance (coverage data, route accuracy): 5/100

Criteria #2: Mode 2 Operation

Rating: 20/100

  • Subjective scoring of Mode 2 mapping presentation.
  • Rating (20/100)
    • UI/UX (Mode 2 presentation, ease of interpretation): 15/100
    • Mode 2 map performance (coverage data, route accuracy): 5/100

Criteria #3: Mode 3 Operation

Rating: 20/100

  • Evaluation of general application functions and compliance with subset of Android Core App Quality Plan. 10/100
  • Subjective scoring of Mode 3 mapping presentation. 10/100
    • UI/UX (Mode 3 presentation, ease of interpretation):
    • Mode 3 map performance (coverage data, route accuracy):

Criteria #4: Final Evaluation — Android Core App Quality

Rating: 20/100

  • Assessment of general application functions and compliance with Android Core App Quality test plan (Pass/Fail for all relevant cases https://developer.android.com/docs/quality-guidelines/core-app-quality).
  • Evaluation will be performed using a commercial Android UE on a supported Android 9 or 10 build. A successful submission will be able to utilize available sensors from a variety of UEs in order to achieve optimal performance.
  • Evaluation of submission summary for compliance with internal requirements (e.g., security, privacy)
  • Rating (20/100)
    • Core App Quality test plan: 10/100
    • General application functionality: 10/100

Criteria #5: Final Evaluation — Application Performance

Rating: 20/100

  • Comprehensive assessment of application performance.
  • Evaluation will be performed using a commercial Android UE on a supported Android 9 or 10 build. A successful submission will be able to utilize available sensors from a variety of UEs in order to achieve optimal performance.
  • Rating (20/100)
    • UI/UX (Modes 1-3 presentation, ease of interpretation): 10/100
    • Modes 1-3 map performance (coverage data, route accuracy): 10/100

3. EXPECTED DELIVERABLES FROM PARTICIPANTS

Review the How to Participate instructions in section 3 of this document to ensure that you are prepared for the in-person Regional Codeathon or Online Contest. The following deliverables will need to be included with the submission:

  • A completed submission form through techtoprotectchallenge.org
  • Software application:
    • Android package (APK file)
  • A 3-minute narrated PowerPoint file or 3-minute narrated video with the following sections:
    • A summary of the submission.
    • Screenshots of maps from each mode.
    • Route maps with descriptions to support map screenshots.
    • Basic information about the participant or participant’s team.
    • Specific requirements of the contest that are being addressed by the participants.
    • Two to four screenshots of the solutions or prototype.
    • Overview of key functions included in the submission.
  • CSV file generated by the application, conforming to the template provided for the challenge.
  • Any available files or links to the developed solution or prototype.
  • Any additional files based on the contest description.
Tech To Protect
Rules & Guidelines