Welcome to Multi-Agent Behaviour Challenge 2022 | 📓Mouse Triplets Baseline

 Looking for teammates? Reply here! | 📻 Join Community Slack Channel

⚔️ Problem statement

In this Task, you will be given a dataset of tracking data of triplets of socially interacting mice. Rather than being asked to detect a specific behavior of interest, we ask you to submit a frame-by-frame representation of the dataset—for example, a low-dimensional embedding of animals' trajectories over time. (For inspiration, you can read about a few existing methods for embedding behavior of individual animals here, here, here, and here.)

To evaluate the quality of your learned representations, we will take a practical approach: we'll use representations as input to train small two layer neural networks on many different "hidden" tasks (each task will have its own neural network), such as detecting occurrence of experimenter-defined actions or distinguishing between two different strains of mice. The goal is therefore to create a representation that captures behavior and generalizes well in any downstream task.

Join our Computational Behavior Slack to discuss the challenge, ask questions, find teammates, or chat with the organizers!

💾 Dataset

We provide frame-by-frame animal pose estimates extracted from top-view videos of trios of interacting mice filmed at 30Hz; raw videos will not be provided for this stage of the competition. Animal poses are characterized by the tracked locations of body parts on each animal, termed "keypoints."

Keypoints are stored in an ndarray with the following properties:

• Dimensions: (# frames) x (animal ID) x (body part) x (x, y coordinate).
• Units: pixels; coordinates are relative to the entire image. Original image dimensions are 850 x 850 for the mouse dataset.

Body parts are ordered: 1) nose, 2) left ear, 3) right ear, 4) neck, 5) left forepaw, 6) right forepaw, 7) center back, 8) left hindpaw, 9) right hindpaw, 10) tail base, 11) tail middle, 12) tail tip.

The placement of these keypoints is illustrated below:

📁 Files

The following files are available in the resources section. A "sequence" is a continuous recording of social interactions between animals: sequences are 1 minute long (1800 frames at 30Hz) in the mouse dataset. The sequence_id is a random hash to anonymize experiment details.

• user_train.npy - Training set for the task, which follows the following schema :

• submission_clips.npy - Test set for the task, which follows the following schema:

• sample_submission.npy - Template for a sample submission for this task, follows the following schema :

In sample_submission, each key in the frame_number_map dictionary refers to the unique sequence id of a video in the test set. The item for each key is expected to be an the start and end index for slicing the embeddings numpy array to get the corresponding embeddings. The embeddings array is a 2D ndarray of floats of size total_frames by X , where X is the dimension of your learned embedding (6 in the above example; maximum permitted embedding dimension is 128), representing the embedded value of each frame in the sequence. total_frames is the sum of all the frames of the sequences, the array should be concatenation of all the embeddings of all the clips.

Public Tasks

To help you evaluate the quality of your embeddings, we provide labels for two sample evaluation subtasks:

1. Chasing is a "frame-level" task, meaning each frame in a 1-minute clip receives a binary label. Chasing frames are those in which any one mouse is pursuing any other; the mice be must within a given distance of each other, and traveling above a given speed, for at least one second. Frames are labeled with a 1 when chasing is detected, and 0 otherwise.
2. Light cycle is a "sequence-level" task, meaning its value is the same for all frames in a sequence. Here, sequences are labeled with a 1 when lights are on, and 0 when lights are off. (Mice are night active, so you will observe more movement when lights are off.)

🚀 Submission

Sample submission format is described in the Files section above.

To test out the system, you can start by uploading the provided sample_submission.npy. When you make your own submissions, they should follow the same format.

🏆 Prizes

The cash prize pool for this task is $3,000 USD total:

• 🥇 1st on leaderboard: $1500 USD
• 🥈 2nd on the leaderboard: $1000 USD
• 🥉 3rd on the leaderboard: $500 USD

📫 Contact

mabe.workshop@gmail.com