💬 Feedback & Suggestions

View post 0

📹 Welcome to Sounding Video Generation Challenge 2024

View post 0

👥 Looking for teammates?

View post 0

👥 Looking for teammates?

View post 0

💬 Feedback & Suggestions

View post 0

📹 Welcome to Sounding Video Generation Challenge 2024

View post 0

📹 Welcome to Sounding Video Generation Challenge 2024

The Sounding Video Generation (SVG) Challenge 2024 is a competition to create AI models that make videos where the visuals match perfectly with sounds, like a dog barking in sync with the video. Participants will work to improve how well sounds and scenes align, with prizes for the best results.

ADVANCING AUDIO-VISUAL SYNCHRONISATION

Join the Sounding Video Generation (SVG) Challenge 2024, a groundbreaking competition at the intersection of video generation and audio-visual synchronisation. This innovative challenge invites participants to build state-of-the-art AI models that generate perfectly aligned and contextually accurate videos guided by audio.

Use your machine learning expertise to advance the frontier of audio-visual synchronisation, transforming datasets into dynamic, synchronised videos. With the right code and creativity, you’ll contribute to a rapidly evolving field that’s set to redefine how we generate and experience multi-modal content.

The Task
Develop models that generate videos with synchronised and contextually relevant audio in two specialised tracks:

• Temporal Alignment Track: Create videos where the audio is perfectly synchronised with the video content in time (e.g., a dog barking exactly when seen in the video). Find the starter kit here.
• Spatial Alignment Track: Develop models that produce videos with spatially aligned audio, creating a real sense of direction and space. Find the starter kit here.

Both tracks aim to push the boundaries of multi-modal AI, offering a unique platform to benchmark cutting-edge solutions in this underexplored domain.

Check out the starter-kits for Temporal Alignment Track and Spatial Alignment Track.
Download the resources for the challenge Temporal Alignment Track and Spatial Alignment Track.

Timeline

• Warmup Round: 29th Oct 2024
• Phase I: 2nd Dec 2024
• Phase II: 3rd Jan 2025
• Challenge End: 25th Mar 2025

Prizes
The challenge boasts a prize pool of USD 35,000 split across both tracks. The top three teams or participants in each track will be rewarded as follows:

• Track 1: Temporal Alignment ($17,500)
  • First place: USD 10,000
  • Second place: USD 5,000
  • Third place: USD 2,500
• Track 2: Spatial Alignment ($17,500)
  • First place: USD 10,000
  • Second place: USD 5,000
  • Third place: USD 2,500

Join a Thriving Community
Collaborate with like-minded researchers, practitioners, and AI enthusiasts eager to share ideas, team up, and drive innovation in this captivating challenge.

Sign up now for the SVG Challenge 2024 and start building your models in the warm-up round!

View post 1

💬 Feedback & Suggestions

We are constantly trying to improve this challenge for you and would appreciate any feedback you might have!

Please reply to this thread with your suggestions and feedback on making the challenge better for you!

• What have been your major pain points so far?
• What would you like to see improved?

All The Best!

View post 0

👥 Looking for teammates?

Competing is more fun with a team!

Introduce yourself here, and find others who are looking to team up!

Format:

• A short introduction about you and your background.
• What brings you to this challenge?
• Some ideas you wish to explore as a part of this challenge?

All The Best!

View post 0

About the Sounding Video Generation (SVG) Challenge 2024 category

View post 0

[Issue resolved] We were asked to submit the revision but revision functionality is disabled

The deadline is now updated on OpenReview too. Please try to make a submission again.

View post 0

Final Evaluation Process & Team Scores

There are no manual scores, only auto-scores.

View post 0

🏆 Meta CRAG Challenge 2024 Winners Announcement

August 02, 2024 11:59PM UTC-0, updated in the post.

View post 0

🏆 Meta CRAG Challenge 2024 Winners Announcement

Hello Participants,

We are excited to announce the winners of the Meta CRAG Challenge 2024. We appreciate your patience as we completed the due diligence process for the annotations.

We want to thank all participants for their efforts and contributions to improve RAG solutions. Over the last four months, the Meta CRAG Challenge has seen over 2000 participants from various countries, making over 5500 submissions. Below are the winners for each task. The final evaluation process can be found here: Final Evaluation Process & Team Scores

Task 1: Retrieval Summarization

• Team db3
• Team md_dh
• Team ElectricSheep
• Simple_w_condition Question: Team dummy_model
• Set Question: Team dummy_model
• Comparison Question: Team dRAGonRAnGers
• Aggregation Question: Team dummy_model
• Multi-hop Question: Team bumblebee7
• Post-processing Question: Team dRAGonRAnGers
• False Premise Question: Team ETSLab

Task 2: Knowledge Graph and Web Retrieval

• Team db3
• Team APEX
• Team md_dh
• Simple_w_condition Question: Team ElectricSheep
• Set Question: Team ElectricSheep
• Comparison Question: Team dRAGonRAnGers
• Aggregation Question: Team ElectricSheep
• Multi-hop Question: Team ElectricSheep
• Post-processing Question: Team ElectricSheep
• False Premise Question: Team Future

Task 3: End-to-End Retrieval-Augmented Generation

• Team db3
• Team APEX
• Team vslyu-team
• Simple_w_condition Question: Team StarTeam
• Set Question: Team md_dh
• Comparison Question: Team dRAGonRAnGers
• Aggregation Question: Team md_dh
• Multi-hop Question: Team ETSLab
• Post-processing Question: Team md_dh
• False Premise Question: Team Riviera4

We welcome all teams to submit a technical report to the 2024 KDD Cup RAG Workshop. The report should follow the KDD submission format.

• Submission link: OpenReview
• Submission deadline: 11:59 UTC, Aug 2, 2024
• Submission format: Submissions are limited to 8 pages (excluding references), must be in PDF, and use the ACM Conference Proceeding template (two-column format). For more details, please refer to: KDD 2024 ADS Track Call for Papers

We extend our sincere gratitude to all participants who contributed to this event!

All the best,
Meta & AIcrowd

View post 2

Final Evaluation Process & Team Scores

Hello Participants,

We want to thank all participants for their efforts and contributions to improving RAG systems. This post provides information about the final evaluation process and shares the final scores.

Manual annotation:

For stage 2 in this challenge, we first conducted auto-evaluation for all the teams that provided submission numbers in Phase 2. We then selected the top 15 teams’ submissions according to the auto-eval scores for manual evaluation. The final scores are determined by the scores calculated from the manual grading labels. Automatic and manual evaluation details can be found in the paper.

Weighting:

We applied traffic weights to the questions to understand the solutions in real-world use cases. The traffic weights come from a real QA use case and are generated as follows. Within each domain, we first clustered the questions into question types with the exact definition of the CRAG questions. Then, we derived the weights for each type based on aggregated data reflective of user interactions. We applied the weight to each CRAG question to bridge the result to reflect user experience and reported the macro average scores across all domains (i.e., giving the same weight to all domains).

Code validation

We conducted a code review for the winning solutions to ensure the validity of the codes. For example, we checked whether there was any prompt attack to mislead the auto-evaluation and whether the solutions used too many hard-coded answers.

The scores from the winning teams are listed below.

The scores from the winning teams are listed below.

We extend our sincere gratitude to all participants who contributed to this event!

All the best,

Meta & AIcrowd

View post 4

Has the Winner Notification already been sent?

My previous reply clearly states that the winner announcement will be made after the due diligence process is completed. Nowhere does it specify that it will be sent only to the winning participants. As is the standard practice for all challenges, the winner announcement email will be sent to all participating teams and candidates once the due diligence process is over.

View post 0

‼️ ⏰ Select Submission ID before 20th June, 2024 23:55 UTC

The winners are not announced yet. You will receive an email about the winner announcement once the due diligence is completed.

View post 0

Has the Winner Notification already been sent?

The winners are not announced yet. You will receive an email about the winner announcement once the due diligence is completed.

View post 1

Change Team Name Request

Your team name has been updated to BMI_DLUT.

View post 0

Change Team Name Request

Hello, Your team name has been updated to AML_LabCityU

View post 0