Skip to main content
Dryad logo

Characterizing long-range search behavior in Diptera using complex 3D virtual environments

Citation

Olsson, Shannon; Kaushik, Pavan Kumar; Renz, Marian (2020), Characterizing long-range search behavior in Diptera using complex 3D virtual environments, Dryad, Dataset, https://doi.org/10.5061/dryad.jq2bvq85s

Abstract

The exemplary search capabilities of flying insects have established them as one of the most diverse taxa on Earth. However, we still lack the fundamental ability to quantify, represent, and predict trajectories under natural contexts to understand search and its applications. For example, flying insects have evolved in complex multimodal 3D environments, but we do not yet understand which features of the natural world are used to locate distant objects. Here, we independently and dynamically manipulate 3D objects, airflow fields, and odor plumes in virtual reality over large spatial and temporal scales. We demonstrate that that flies make use of features such as foreground segmentation, perspective, motion parallax, and integration of multiple modalities to navigate to objects in a complex 3D landscape while in flight. We first show that tethered flying insects of multiple species navigate to virtual 3D objects. Using the apple fly, Rhagoletis pomonella, we then measure their reactive distance to objects and show that these flies use perspective and local parallax cues to distinguish and navigate to virtual objects of different sizes and distances. We also show that apple flies can orient in the absence of optic flow by using only directional airflow cues, and require simultaneous odor and directional airflow input for plume following to a host volatile blend. The elucidation of these features unlocks the opportunity to quantify parameters underlying insect behavior such as reactive space, optimal foraging, and dispersal, as well as develop strategies for pest management, pollination, robotics and search algorithms.

Methods

We assessed critical parameters of long range search including motion parallax, perspective, reactive distance, anemotaxis, and plume following using a multimodal virtual reality arena (MultiMoVR). To this end, we provided photorealistic scenes and perspective-accurate stimuli of 3D tree models along with grass and sky textures in a 1025 m x 1025 m landscape, including directional airflow and odor. This landscape was presented in a periodic boundary condition such that as the animal approaches the end of the virtual landscape, it is seamlessly teleported to the opposite side, so the animal can essentially translate infinitely in any direction. Using this arena, we show that R. pomonella can approach and discriminate virtual objects of varying sizes and distances in a complex 3D environment, respond to directional airflow based on velocity, and orient to directional odor flux in VR. We also show that multiple Dipteran species, including a North American pest (R. pomonella), a tropical vector (Aedes aegypti), an Asian species (Pselliophora laeta) and a cosmopolitan pollinator (Eristalis tenax) can navigate towards virtual objects at distances found in nature using this system.

Usage Notes

INCLUDED FILES:

ARENA FILES:

MultiMoVR Setup.zip:  Design files and software installation guide for MultiMoVR setup.

DATA FILES:

FILE NAME FILE DESCRIPTION
mos_vision.h5 Mosquito to tree
hover_vision.h5 Hover fly to flower
crane_vision.h5 Crane fly to tree
apple_wind.h5 Apple fly to wind 
apple_odor.h5 Apple fly to odor
apple_motionParallax.h5 Apple fly to motion parallax trees
apple_vision.h5 Apple fly to tree

DESCRIPTION OF COLUMNS FOR EACH DATA FILE:

COLUMN NAME COLUMN DESCRIPTION
trajectory__DCoffset The DC offset to adjust for side bias in tethering and alignment
trajectory__boutFrame Frame number of a given trial
trajectory__case The condition of the trial. No tree, tree of left, tree on right, tree on both sides
trajectory__gain The gain, in rad/deg/frame. Multiple by 2.89 to get it in conventional units (deg/deg/s)
trajectory__headingControl True when in closed loop for heading
trajectory__o1Pos_x Object1 pos x
trajectory__o1Pos_y Object1 pos y
trajectory__o1Pos_z Object1 pos z
trajectory__o2Pos_x Object2 pos x
trajectory__o2Pos_y Object2 pos y
trajectory__o2Pos_z Object2 pos z
trajectory__pOri_x Insect heading
trajectory__pOri_y Insect pitch
trajectory__pOri_z Insect roll
trajectory__pPos_x Insect pos x
trajectory__pPos_y Insect pos y
trajectory__pPos_z Insect pos z
trajectory__pfStimState State of odor valve
trajectory__reset True when change of trial
trajectory__runNum Run number for an entire set of all case trials
trajectory__speed Insect speed
trajectory__speedControl True when in closed loop for speed control
trajectory__valve1 Valve1 odor bottle
trajectory__valve2 Valve2 odor bottle
trajectory__valve3 Valve3 odor bottle
trajectory__wbad Wing Beat Amplitude Difference, L-R, in radians
trajectory__wbas Wing Beat Amplitude Sum, L+R, in radians

Funding

Science and Engineering Research Board, Award: Ramanujan Fellowship

Microsoft Research, Award: Shannon Olsson

Tata Institute of Fundamental Research, Award: Graduate Research Fellowship

Hochschule Bremen, Award: Internship Fellowship