Smart 3D Cameras

Work in Progress

Technology Roadmap Sections and Deliverables

The Smart 3D Camera roadmap is a level 2 roadmap as it enables the level 1 roadmaps for autonomous navigation of robots, drones and cars.

• 2S3DC - Smart 3D Camera

Roadmap Overview

Smart 3D Cameras use a pair of identical optical imaging sensors and IR projectors (in certain use-cases) to capture stereo images of the environment. These images are then processed to calculate the disparity and then extract depth information for all pixels. In addition to the depth map, the scene is segmented to extract objects of interest and to identify them using training neural nets. Note that this roadmap will focus on passive stereo vision cameras that DO NOT use structured light.

Design Structure Matrix (DSM) Allocation

The 2S3DCAM roadmap is part of the larger company effort to develop an autonomous navigation stack as it enables 1ANAV.

The following tree can be discerned: 1ANAV

• 2S3DCAM
  • 3OPTFIL

Roadmap Model using OPM

Figures of Merit

Figure of Merit	Units	Description
Million Disparity Estimations Per Second (MDE/s) (10^6)	-	Comparison metric defined as: MDE/s = Image resolution * disparities * sensing speed
Power Consumption	Watts (W)	Power consumed by the entire stereo camera and image processing pipeline to produce a depth map
Image resolution	Pixel (px)	Number of pixels in the captured image
Range (m)	m	The maximum sensing distance
Accuracy (m)	m	The measuring confidence in the depth data point
Sensing Speed (fps or Hz)	fps or Hz	The scanning frame rate of the entire system
Depth Pixels (px)	px	The number of data points in the generated depth map
Cost ($)	$	The commercial price for a customer, at volume
Energy Consumed per Depth Pixel, E_dpx	W/px	The total power consumed by the sensing and processing pipeline divided by the number of depth pixels

Besides defining what the FOMs are, this section of the roadmap should also contain the FOM trends over time dFOM/dt as well as some of the key governing equations that underpin the technology. These governing equations can be derived from physics (or chemistry, biology ..) or they can be empirically derived from a multivariate regression model. The table below shows an example of a key governing equation governing (solar-) electric aircraft.

Alignment with Company Strategic Drivers

#	Strategic Driver	Alignment and Targets
1	To develop a compact, high performance and low-power smart 3D camera that can detect objects in both indoors and outdoor environments	The 2S3DCAM roadmap will target the development of a passive stereo camera with onboard computing that has a sensing range of >20m, sensing speed of >30fps at an energy cost lower than 1mW/px in a 15cm x 5cm x 5cm footprint.
2	To enable autonomous classification and identification of relevant objects in the scene	The 2S3DCAM roadmap will enable the capability for AI neural nets to run onboard the camera to perform image classification and recognition actions.

Positioning of Company vs. Competition

Technical Model

In order to assess the feasibility of technical (and financial) targets at the level of the 2SEA roadmap it is necessary to develop a technical model. The purpose of such a model is to explore the design tradespace and establish what are the active constraints in the system. The first step can be to establish a morphological matrix that shows the main technology selection alternatives that exist at the first level of decomposition, see the figure below.

It is interesting to note that the architecture and technology selections for the three aircraft (Zephyr, Solar Impulse 2 and E-Fan 2.0) are quite different. While Zephyr uses lithium-sulfur batteries, the other two use the more conventional lithium-ion batteries. Solar Impulse uses the less efficient (but more affordable) single cell silicon-based PV, while Zephyr uses specially manufactured thin film multi-junction cells and so forth.

The technical model centers on the E-range and E-endurance equations and compares different aircraft sizing (e.g. wing span, engine power, battery capacity) taking into account aerodynamics, weights and balance, the performance of the aircraft and also its manufacturing cost. It is important to use Multidisciplinary Design Optimization (MDO) when selecting and sizing technologies in order to get the most out of them and compare them fairly (see below).

Financial Model

The figure below contains a sample NPV analysis underlying the 2SEA roadmap. It shows the non-recurring cost (NRC) of the product development project (PDP), which includes the R&D expenditures as negative numbers. A ramp up-period of 4 years is planned with a flat revenue plateau (of 400 million per year) and a total program duration of 24 years.

List of R&T Projects and Prototypes

In order to select and prioritize R&D (R&T) projects we recommend using the technical and financial models developed as part of the roadmap to rank-order projects based on an objective set of criteria and analysis. The figure below illustrates how technical models are used to make technology project selections, e.g based on the previously stated 2030 target performance and Figure 8-17 (see the Chapter 8 of the text) shows the outcome if none of the three potential projects are selected.

Key Publications, Presentations and Patents

A good technology roadmap should contain a comprehensive list of publications, presentations and key patents as shown in Figure 8-19. This includes literature trends, papers published at key conferences and in the trade literature and trade press.

Technology Strategy Statement

A technology roadmap should conclude and be summarized by both a written statement that summarizes the technology strategy coming out of the roadmap as well as a graphic that shows the key R&D investments, targets and a vision for this technology (and associated product or service) over time. For the 2SEA roadmap the statement could read as follows:

Our target is to develop a new solar-powered and electrically-driven UAV as a HAPS service platform with an Entry-into-Service date of 2030. To achieve the target of an endurance of 500 days and useful payload of 10 kg we will invest in two R&D projects. The first is a flight demonstrator with a first flight by 2027 to demonstrate a full-year aloft (365 days) at an equatorial latitude with a payload of 10 kg. The second project is an accelerated development of Li-S batteries with our partner XYZ with a target lifetime performance of 500 charge-discharge cycles by 2027. This is an enabling technology to reach our 2030 technical and business targets.